2023-02-07 15:53:49,452 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-07 15:53:49,684 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed 2023-02-07 15:53:49,696 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-07 15:53:49,697 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a, deleteOnExit=true 2023-02-07 15:53:49,697 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-07 15:53:49,697 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/test.cache.data in system properties and HBase conf 2023-02-07 15:53:49,698 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/hadoop.tmp.dir in system properties and HBase conf 2023-02-07 15:53:49,698 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/hadoop.log.dir in system properties and HBase conf 2023-02-07 15:53:49,699 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-07 15:53:49,699 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-07 15:53:49,699 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-07 15:53:49,805 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-07 15:53:50,148 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-07 15:53:50,153 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-07 15:53:50,153 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-07 15:53:50,153 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-07 15:53:50,154 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-07 15:53:50,154 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-07 15:53:50,154 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-07 15:53:50,155 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-07 15:53:50,155 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-07 15:53:50,155 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-07 15:53:50,156 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/nfs.dump.dir in system properties and HBase conf 2023-02-07 15:53:50,156 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/java.io.tmpdir in system properties and HBase conf 2023-02-07 15:53:50,157 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-07 15:53:50,157 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-07 15:53:50,158 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-07 15:53:50,590 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-07 15:53:50,594 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-07 15:53:51,113 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-07 15:53:51,287 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-07 15:53:51,303 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-07 15:53:51,337 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-07 15:53:51,365 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/java.io.tmpdir/Jetty_localhost_localdomain_41027_hdfs____.pe2z4e/webapp 2023-02-07 15:53:51,500 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:41027 2023-02-07 15:53:51,509 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-07 15:53:51,509 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-07 15:53:52,182 WARN [Listener at localhost.localdomain/43555] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-07 15:53:52,274 WARN [Listener at localhost.localdomain/43555] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-07 15:53:52,291 WARN [Listener at localhost.localdomain/43555] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-07 15:53:52,298 INFO [Listener at localhost.localdomain/43555] log.Slf4jLog(67): jetty-6.1.26 2023-02-07 15:53:52,303 INFO [Listener at localhost.localdomain/43555] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/java.io.tmpdir/Jetty_localhost_36451_datanode____.6tw1e4/webapp 2023-02-07 15:53:52,380 INFO [Listener at localhost.localdomain/43555] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36451 2023-02-07 15:53:52,674 WARN [Listener at localhost.localdomain/39775] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-07 15:53:52,685 WARN [Listener at localhost.localdomain/39775] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-07 15:53:52,689 WARN [Listener at localhost.localdomain/39775] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-07 15:53:52,691 INFO [Listener at localhost.localdomain/39775] log.Slf4jLog(67): jetty-6.1.26 2023-02-07 15:53:52,697 INFO [Listener at localhost.localdomain/39775] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/java.io.tmpdir/Jetty_localhost_37241_datanode____e958ky/webapp 2023-02-07 15:53:52,799 INFO [Listener at localhost.localdomain/39775] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37241 2023-02-07 15:53:52,810 WARN [Listener at localhost.localdomain/45001] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-07 15:53:52,825 WARN [Listener at localhost.localdomain/45001] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-07 15:53:52,828 WARN [Listener at localhost.localdomain/45001] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-07 15:53:52,830 INFO [Listener at localhost.localdomain/45001] log.Slf4jLog(67): jetty-6.1.26 2023-02-07 15:53:52,839 INFO [Listener at localhost.localdomain/45001] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/java.io.tmpdir/Jetty_localhost_38563_datanode____.gum6b0/webapp 2023-02-07 15:53:52,924 INFO [Listener at localhost.localdomain/45001] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38563 2023-02-07 15:53:52,937 WARN [Listener at localhost.localdomain/41965] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-07 15:53:54,656 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd632ff07bcadebbb: Processing first storage report for DS-4c327e91-f723-45ab-baa6-5e10ff9a92ea from datanode bb8a498b-c978-4fc3-b7cd-17da28adf753 2023-02-07 15:53:54,658 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd632ff07bcadebbb: from storage DS-4c327e91-f723-45ab-baa6-5e10ff9a92ea node DatanodeRegistration(127.0.0.1:41397, datanodeUuid=bb8a498b-c978-4fc3-b7cd-17da28adf753, infoPort=35355, infoSecurePort=0, ipcPort=41965, storageInfo=lv=-57;cid=testClusterID;nsid=335937489;c=1675785230655), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-07 15:53:54,658 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x2c9ccde6cd4565ac: Processing first storage report for DS-0bc155ad-6b7d-4aab-8f0f-157a0fc146e9 from datanode 7f437b77-6c8e-4bcb-8556-91fa3c36506b 2023-02-07 15:53:54,658 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x2c9ccde6cd4565ac: from storage DS-0bc155ad-6b7d-4aab-8f0f-157a0fc146e9 node DatanodeRegistration(127.0.0.1:41793, datanodeUuid=7f437b77-6c8e-4bcb-8556-91fa3c36506b, infoPort=38419, infoSecurePort=0, ipcPort=39775, storageInfo=lv=-57;cid=testClusterID;nsid=335937489;c=1675785230655), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-07 15:53:54,658 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x45e31f028684837f: Processing first storage report for DS-f1e4096b-8cd7-4760-a5b7-364d59fe273e from datanode 8f3909c3-b4fb-4933-8c65-751b48eaa405 2023-02-07 15:53:54,658 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x45e31f028684837f: from storage DS-f1e4096b-8cd7-4760-a5b7-364d59fe273e node DatanodeRegistration(127.0.0.1:41359, datanodeUuid=8f3909c3-b4fb-4933-8c65-751b48eaa405, infoPort=45749, infoSecurePort=0, ipcPort=45001, storageInfo=lv=-57;cid=testClusterID;nsid=335937489;c=1675785230655), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-07 15:53:54,658 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd632ff07bcadebbb: Processing first storage report for DS-ead5e13c-d901-424f-a76b-14ac1b10aafd from datanode bb8a498b-c978-4fc3-b7cd-17da28adf753 2023-02-07 15:53:54,658 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd632ff07bcadebbb: from storage DS-ead5e13c-d901-424f-a76b-14ac1b10aafd node DatanodeRegistration(127.0.0.1:41397, datanodeUuid=bb8a498b-c978-4fc3-b7cd-17da28adf753, infoPort=35355, infoSecurePort=0, ipcPort=41965, storageInfo=lv=-57;cid=testClusterID;nsid=335937489;c=1675785230655), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-07 15:53:54,659 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x2c9ccde6cd4565ac: Processing first storage report for DS-d17df42c-3e78-4453-9ed3-8271ec366563 from datanode 7f437b77-6c8e-4bcb-8556-91fa3c36506b 2023-02-07 15:53:54,659 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x2c9ccde6cd4565ac: from storage DS-d17df42c-3e78-4453-9ed3-8271ec366563 node DatanodeRegistration(127.0.0.1:41793, datanodeUuid=7f437b77-6c8e-4bcb-8556-91fa3c36506b, infoPort=38419, infoSecurePort=0, ipcPort=39775, storageInfo=lv=-57;cid=testClusterID;nsid=335937489;c=1675785230655), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-07 15:53:54,659 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x45e31f028684837f: Processing first storage report for DS-3e4eb20e-a095-4ba7-bcd0-72af193cb3ba from datanode 8f3909c3-b4fb-4933-8c65-751b48eaa405 2023-02-07 15:53:54,659 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x45e31f028684837f: from storage DS-3e4eb20e-a095-4ba7-bcd0-72af193cb3ba node DatanodeRegistration(127.0.0.1:41359, datanodeUuid=8f3909c3-b4fb-4933-8c65-751b48eaa405, infoPort=45749, infoSecurePort=0, ipcPort=45001, storageInfo=lv=-57;cid=testClusterID;nsid=335937489;c=1675785230655), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-07 15:53:54,751 DEBUG [Listener at localhost.localdomain/41965] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed 2023-02-07 15:53:54,807 INFO [Listener at localhost.localdomain/41965] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/zookeeper_0, clientPort=51095, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-07 15:53:54,821 INFO [Listener at localhost.localdomain/41965] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=51095 2023-02-07 15:53:54,831 INFO [Listener at localhost.localdomain/41965] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 15:53:54,834 INFO [Listener at localhost.localdomain/41965] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 15:53:55,471 INFO [Listener at localhost.localdomain/41965] util.FSUtils(479): Created version file at hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8 with version=8 2023-02-07 15:53:55,471 INFO [Listener at localhost.localdomain/41965] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/hbase-staging 2023-02-07 15:53:55,747 INFO [Listener at localhost.localdomain/41965] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-07 15:53:56,137 INFO [Listener at localhost.localdomain/41965] client.ConnectionUtils(127): master/jenkins-hbase12:0 server-side Connection retries=45 2023-02-07 15:53:56,164 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 15:53:56,165 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-07 15:53:56,165 INFO [Listener at localhost.localdomain/41965] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-07 15:53:56,165 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 15:53:56,165 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-07 15:53:56,286 INFO [Listener at localhost.localdomain/41965] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-07 15:53:56,345 DEBUG [Listener at localhost.localdomain/41965] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-07 15:53:56,434 INFO [Listener at localhost.localdomain/41965] ipc.NettyRpcServer(120): Bind to /136.243.104.168:37073 2023-02-07 15:53:56,443 INFO [Listener at localhost.localdomain/41965] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 15:53:56,444 INFO [Listener at localhost.localdomain/41965] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 15:53:56,461 INFO [Listener at localhost.localdomain/41965] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37073 connecting to ZooKeeper ensemble=127.0.0.1:51095 2023-02-07 15:53:56,626 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:370730x0, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-07 15:53:56,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:37073-0x1013e2134d40000 connected 2023-02-07 15:53:56,720 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-07 15:53:56,721 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 15:53:56,724 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-07 15:53:56,730 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37073 2023-02-07 15:53:56,731 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37073 2023-02-07 15:53:56,731 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37073 2023-02-07 15:53:56,731 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37073 2023-02-07 15:53:56,732 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37073 2023-02-07 15:53:56,737 INFO [Listener at localhost.localdomain/41965] master.HMaster(439): hbase.rootdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8, hbase.cluster.distributed=false 2023-02-07 15:53:56,798 INFO [Listener at localhost.localdomain/41965] client.ConnectionUtils(127): regionserver/jenkins-hbase12:0 server-side Connection retries=45 2023-02-07 15:53:56,798 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 15:53:56,798 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-07 15:53:56,798 INFO [Listener at localhost.localdomain/41965] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-07 15:53:56,798 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 15:53:56,798 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-07 15:53:56,802 INFO [Listener at localhost.localdomain/41965] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-07 15:53:56,806 INFO [Listener at localhost.localdomain/41965] ipc.NettyRpcServer(120): Bind to /136.243.104.168:38629 2023-02-07 15:53:56,809 INFO [Listener at localhost.localdomain/41965] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-07 15:53:56,815 DEBUG [Listener at localhost.localdomain/41965] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-07 15:53:56,816 INFO [Listener at localhost.localdomain/41965] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 15:53:56,818 INFO [Listener at localhost.localdomain/41965] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 15:53:56,820 INFO [Listener at localhost.localdomain/41965] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38629 connecting to ZooKeeper ensemble=127.0.0.1:51095 2023-02-07 15:53:56,834 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:386290x0, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-07 15:53:56,836 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:38629-0x1013e2134d40001 connected 2023-02-07 15:53:56,836 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-07 15:53:56,838 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 15:53:56,839 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-07 15:53:56,841 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38629 2023-02-07 15:53:56,841 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38629 2023-02-07 15:53:56,842 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38629 2023-02-07 15:53:56,842 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38629 2023-02-07 15:53:56,843 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38629 2023-02-07 15:53:56,854 INFO [Listener at localhost.localdomain/41965] client.ConnectionUtils(127): regionserver/jenkins-hbase12:0 server-side Connection retries=45 2023-02-07 15:53:56,855 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 15:53:56,855 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-07 15:53:56,856 INFO [Listener at localhost.localdomain/41965] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-07 15:53:56,856 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 15:53:56,856 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-07 15:53:56,857 INFO [Listener at localhost.localdomain/41965] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-07 15:53:56,861 INFO [Listener at localhost.localdomain/41965] ipc.NettyRpcServer(120): Bind to /136.243.104.168:35657 2023-02-07 15:53:56,861 INFO [Listener at localhost.localdomain/41965] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-07 15:53:56,862 DEBUG [Listener at localhost.localdomain/41965] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-07 15:53:56,863 INFO [Listener at localhost.localdomain/41965] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 15:53:56,866 INFO [Listener at localhost.localdomain/41965] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 15:53:56,869 INFO [Listener at localhost.localdomain/41965] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35657 connecting to ZooKeeper ensemble=127.0.0.1:51095 2023-02-07 15:53:56,882 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:356570x0, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-07 15:53:56,884 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:35657-0x1013e2134d40002 connected 2023-02-07 15:53:56,884 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-07 15:53:56,885 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 15:53:56,886 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-07 15:53:56,886 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35657 2023-02-07 15:53:56,887 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35657 2023-02-07 15:53:56,887 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35657 2023-02-07 15:53:56,889 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35657 2023-02-07 15:53:56,889 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35657 2023-02-07 15:53:56,899 INFO [Listener at localhost.localdomain/41965] client.ConnectionUtils(127): regionserver/jenkins-hbase12:0 server-side Connection retries=45 2023-02-07 15:53:56,899 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 15:53:56,899 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-07 15:53:56,899 INFO [Listener at localhost.localdomain/41965] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-07 15:53:56,900 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 15:53:56,900 INFO [Listener at localhost.localdomain/41965] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-07 15:53:56,900 INFO [Listener at localhost.localdomain/41965] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-07 15:53:56,901 INFO [Listener at localhost.localdomain/41965] ipc.NettyRpcServer(120): Bind to /136.243.104.168:37399 2023-02-07 15:53:56,902 INFO [Listener at localhost.localdomain/41965] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-07 15:53:56,905 DEBUG [Listener at localhost.localdomain/41965] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-07 15:53:56,907 INFO [Listener at localhost.localdomain/41965] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 15:53:56,908 INFO [Listener at localhost.localdomain/41965] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 15:53:56,909 INFO [Listener at localhost.localdomain/41965] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37399 connecting to ZooKeeper ensemble=127.0.0.1:51095 2023-02-07 15:53:56,919 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:373990x0, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-07 15:53:56,920 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ZKUtil(164): regionserver:373990x0, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-07 15:53:56,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:37399-0x1013e2134d40003 connected 2023-02-07 15:53:56,921 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 15:53:56,921 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-07 15:53:56,922 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37399 2023-02-07 15:53:56,923 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37399 2023-02-07 15:53:56,923 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37399 2023-02-07 15:53:56,925 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37399 2023-02-07 15:53:56,927 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37399 2023-02-07 15:53:56,928 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase12.apache.org,37073,1675785235583 2023-02-07 15:53:56,946 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-07 15:53:56,948 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase12.apache.org,37073,1675785235583 2023-02-07 15:53:56,971 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-07 15:53:56,971 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-07 15:53:56,971 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-07 15:53:56,971 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-07 15:53:56,971 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 15:53:56,973 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-07 15:53:56,974 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase12.apache.org,37073,1675785235583 from backup master directory 2023-02-07 15:53:56,974 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-07 15:53:56,988 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase12.apache.org,37073,1675785235583 2023-02-07 15:53:56,988 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-07 15:53:56,988 WARN [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-07 15:53:56,989 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase12.apache.org,37073,1675785235583 2023-02-07 15:53:56,991 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-07 15:53:56,992 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-07 15:53:57,087 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/hbase.id with ID: 547a5484-1d77-4871-9984-cfc06cab62f8 2023-02-07 15:53:57,124 INFO [master/jenkins-hbase12:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 15:53:57,156 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 15:53:57,208 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x79800e45 to 127.0.0.1:51095 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-07 15:53:57,247 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77a2df34, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-07 15:53:57,267 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-07 15:53:57,269 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-07 15:53:57,283 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-07 15:53:57,283 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-07 15:53:57,285 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-07 15:53:57,288 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-07 15:53:57,290 INFO [master/jenkins-hbase12:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-07 15:53:57,319 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/data/master/store-tmp 2023-02-07 15:53:57,380 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:53:57,381 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-07 15:53:57,381 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 15:53:57,381 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 15:53:57,381 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-07 15:53:57,381 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 15:53:57,381 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 15:53:57,382 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675785237380Disabling compacts and flushes for region at 1675785237381 (+1 ms)Disabling writes for close at 1675785237381Writing region close event to WAL at 1675785237381Closed at 1675785237381 2023-02-07 15:53:57,383 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/WALs/jenkins-hbase12.apache.org,37073,1675785235583 2023-02-07 15:53:57,405 INFO [master/jenkins-hbase12:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C37073%2C1675785235583, suffix=, logDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/WALs/jenkins-hbase12.apache.org,37073,1675785235583, archiveDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/oldWALs, maxLogs=10 2023-02-07 15:53:57,464 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41793,DS-0bc155ad-6b7d-4aab-8f0f-157a0fc146e9,DISK] 2023-02-07 15:53:57,464 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41397,DS-4c327e91-f723-45ab-baa6-5e10ff9a92ea,DISK] 2023-02-07 15:53:57,464 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41359,DS-f1e4096b-8cd7-4760-a5b7-364d59fe273e,DISK] 2023-02-07 15:53:57,474 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-07 15:53:57,546 INFO [master/jenkins-hbase12:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/WALs/jenkins-hbase12.apache.org,37073,1675785235583/jenkins-hbase12.apache.org%2C37073%2C1675785235583.1675785237414 2023-02-07 15:53:57,547 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41359,DS-f1e4096b-8cd7-4760-a5b7-364d59fe273e,DISK], DatanodeInfoWithStorage[127.0.0.1:41397,DS-4c327e91-f723-45ab-baa6-5e10ff9a92ea,DISK], DatanodeInfoWithStorage[127.0.0.1:41793,DS-0bc155ad-6b7d-4aab-8f0f-157a0fc146e9,DISK]] 2023-02-07 15:53:57,547 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:53:57,547 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:53:57,550 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-07 15:53:57,551 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-07 15:53:57,603 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-07 15:53:57,611 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-07 15:53:57,636 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-07 15:53:57,648 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:53:57,654 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-07 15:53:57,656 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-07 15:53:57,674 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-07 15:53:57,679 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:53:57,680 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68463303, jitterRate=0.020182713866233826}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-07 15:53:57,680 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675785237574Initializing all the Stores at 1675785237576 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785237576Cleaning up temporary data from old regions at 1675785237664 (+88 ms)Cleaning up detritus from prior splits at 1675785237665 (+1 ms)Region opened successfully at 1675785237680 (+15 ms) 2023-02-07 15:53:57,682 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-07 15:53:57,700 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:53:57,700 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:53:57,702 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:53:57,704 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 15:53:57,733 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 28 msec 2023-02-07 15:53:57,733 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-07 15:53:57,756 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-07 15:53:57,761 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-07 15:53:57,783 INFO [master/jenkins-hbase12:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-07 15:53:57,787 INFO [master/jenkins-hbase12:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-07 15:53:57,788 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-07 15:53:57,793 INFO [master/jenkins-hbase12:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-07 15:53:57,797 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-07 15:53:57,851 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 15:53:57,852 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-07 15:53:57,853 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-07 15:53:57,865 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-07 15:53:57,876 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-07 15:53:57,876 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-07 15:53:57,876 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-07 15:53:57,876 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-07 15:53:57,876 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 15:53:57,877 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase12.apache.org,37073,1675785235583, sessionid=0x1013e2134d40000, setting cluster-up flag (Was=false) 2023-02-07 15:53:57,908 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 15:53:57,939 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-07 15:53:57,941 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase12.apache.org,37073,1675785235583 2023-02-07 15:53:57,966 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 15:53:57,998 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-07 15:53:58,000 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase12.apache.org,37073,1675785235583 2023-02-07 15:53:58,003 WARN [master/jenkins-hbase12:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.hbase-snapshot/.tmp 2023-02-07 15:53:58,031 INFO [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(952): ClusterId : 547a5484-1d77-4871-9984-cfc06cab62f8 2023-02-07 15:53:58,031 INFO [RS:1;jenkins-hbase12:35657] regionserver.HRegionServer(952): ClusterId : 547a5484-1d77-4871-9984-cfc06cab62f8 2023-02-07 15:53:58,031 INFO [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(952): ClusterId : 547a5484-1d77-4871-9984-cfc06cab62f8 2023-02-07 15:53:58,036 DEBUG [RS:0;jenkins-hbase12:38629] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-07 15:53:58,036 DEBUG [RS:1;jenkins-hbase12:35657] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-07 15:53:58,036 DEBUG [RS:2;jenkins-hbase12:37399] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-07 15:53:58,063 DEBUG [RS:1;jenkins-hbase12:35657] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-07 15:53:58,063 DEBUG [RS:2;jenkins-hbase12:37399] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-07 15:53:58,063 DEBUG [RS:0;jenkins-hbase12:38629] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-07 15:53:58,063 DEBUG [RS:2;jenkins-hbase12:37399] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-07 15:53:58,063 DEBUG [RS:1;jenkins-hbase12:35657] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-07 15:53:58,063 DEBUG [RS:0;jenkins-hbase12:38629] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-07 15:53:58,087 DEBUG [RS:0;jenkins-hbase12:38629] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-07 15:53:58,087 DEBUG [RS:2;jenkins-hbase12:37399] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-07 15:53:58,087 DEBUG [RS:1;jenkins-hbase12:35657] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-07 15:53:58,090 DEBUG [RS:2;jenkins-hbase12:37399] zookeeper.ReadOnlyZKClient(139): Connect 0x738961ee to 127.0.0.1:51095 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-07 15:53:58,090 DEBUG [RS:0;jenkins-hbase12:38629] zookeeper.ReadOnlyZKClient(139): Connect 0x38ca98b9 to 127.0.0.1:51095 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-07 15:53:58,090 DEBUG [RS:1;jenkins-hbase12:35657] zookeeper.ReadOnlyZKClient(139): Connect 0x4474c8fd to 127.0.0.1:51095 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-07 15:53:58,115 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-07 15:53:58,119 DEBUG [RS:2;jenkins-hbase12:37399] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7d4e81ef, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-07 15:53:58,120 DEBUG [RS:0;jenkins-hbase12:38629] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@360d71d0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-07 15:53:58,120 DEBUG [RS:1;jenkins-hbase12:35657] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4ab33e03, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-07 15:53:58,120 DEBUG [RS:0;jenkins-hbase12:38629] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3232f3c9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-02-07 15:53:58,120 DEBUG [RS:2;jenkins-hbase12:37399] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79710a9e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-02-07 15:53:58,120 DEBUG [RS:1;jenkins-hbase12:35657] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e8ca38d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-02-07 15:53:58,127 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-02-07 15:53:58,127 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-02-07 15:53:58,127 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-02-07 15:53:58,127 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-02-07 15:53:58,127 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase12:0, corePoolSize=10, maxPoolSize=10 2023-02-07 15:53:58,127 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,127 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-02-07 15:53:58,128 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,128 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785268128 2023-02-07 15:53:58,130 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-07 15:53:58,135 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-07 15:53:58,136 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-07 15:53:58,141 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-07 15:53:58,142 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-07 15:53:58,144 DEBUG [RS:0;jenkins-hbase12:38629] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase12:38629 2023-02-07 15:53:58,145 DEBUG [RS:2;jenkins-hbase12:37399] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase12:37399 2023-02-07 15:53:58,147 DEBUG [RS:1;jenkins-hbase12:35657] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase12:35657 2023-02-07 15:53:58,150 INFO [RS:0;jenkins-hbase12:38629] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-07 15:53:58,150 INFO [RS:0;jenkins-hbase12:38629] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-07 15:53:58,150 DEBUG [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(1023): About to register with Master. 2023-02-07 15:53:58,151 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-07 15:53:58,151 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-07 15:53:58,152 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-07 15:53:58,152 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-07 15:53:58,152 INFO [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase12.apache.org,37073,1675785235583 with isa=jenkins-hbase12.apache.org/136.243.104.168:38629, startcode=1675785236797 2023-02-07 15:53:58,150 INFO [RS:2;jenkins-hbase12:37399] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-07 15:53:58,150 INFO [RS:1;jenkins-hbase12:35657] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-07 15:53:58,153 INFO [RS:1;jenkins-hbase12:35657] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-07 15:53:58,153 INFO [RS:2;jenkins-hbase12:37399] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-07 15:53:58,154 DEBUG [RS:1;jenkins-hbase12:35657] regionserver.HRegionServer(1023): About to register with Master. 2023-02-07 15:53:58,153 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,154 DEBUG [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(1023): About to register with Master. 2023-02-07 15:53:58,155 INFO [RS:1;jenkins-hbase12:35657] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase12.apache.org,37073,1675785235583 with isa=jenkins-hbase12.apache.org/136.243.104.168:35657, startcode=1675785236854 2023-02-07 15:53:58,155 INFO [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase12.apache.org,37073,1675785235583 with isa=jenkins-hbase12.apache.org/136.243.104.168:37399, startcode=1675785236898 2023-02-07 15:53:58,158 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-07 15:53:58,159 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-07 15:53:58,160 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-07 15:53:58,162 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-07 15:53:58,162 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-07 15:53:58,165 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.large.0-1675785238163,5,FailOnTimeoutGroup] 2023-02-07 15:53:58,165 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.small.0-1675785238165,5,FailOnTimeoutGroup] 2023-02-07 15:53:58,165 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,165 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-07 15:53:58,167 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,167 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,173 DEBUG [RS:1;jenkins-hbase12:35657] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-07 15:53:58,173 DEBUG [RS:2;jenkins-hbase12:37399] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-07 15:53:58,173 DEBUG [RS:0;jenkins-hbase12:38629] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-07 15:53:58,198 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-07 15:53:58,199 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-07 15:53:58,199 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8 2023-02-07 15:53:58,239 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:53:58,244 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-07 15:53:58,247 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/info 2023-02-07 15:53:58,248 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-07 15:53:58,248 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:53:58,249 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-07 15:53:58,252 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/rep_barrier 2023-02-07 15:53:58,253 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-07 15:53:58,254 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:53:58,254 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-07 15:53:58,255 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:37759, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-07 15:53:58,255 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:60107, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-07 15:53:58,255 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:35639, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-07 15:53:58,257 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/table 2023-02-07 15:53:58,258 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-07 15:53:58,259 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:53:58,263 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740 2023-02-07 15:53:58,264 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740 2023-02-07 15:53:58,269 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-07 15:53:58,271 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37073] master.ServerManager(394): Registering regionserver=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:53:58,271 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37073] master.ServerManager(394): Registering regionserver=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:53:58,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.ServerManager(394): Registering regionserver=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:53:58,274 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-07 15:53:58,278 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:53:58,279 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68479341, jitterRate=0.020421698689460754}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-07 15:53:58,279 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675785238240Initializing all the Stores at 1675785238243 (+3 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675785238243Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785238243Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675785238243Cleaning up temporary data from old regions at 1675785238266 (+23 ms)Cleaning up detritus from prior splits at 1675785238267 (+1 ms)Region opened successfully at 1675785238279 (+12 ms) 2023-02-07 15:53:58,279 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-07 15:53:58,279 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-07 15:53:58,279 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-07 15:53:58,280 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-07 15:53:58,280 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-07 15:53:58,281 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-07 15:53:58,281 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675785238279Disabling compacts and flushes for region at 1675785238279Disabling writes for close at 1675785238280 (+1 ms)Writing region close event to WAL at 1675785238280Closed at 1675785238280 2023-02-07 15:53:58,289 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-07 15:53:58,289 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-07 15:53:58,292 DEBUG [RS:1;jenkins-hbase12:35657] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8 2023-02-07 15:53:58,293 DEBUG [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8 2023-02-07 15:53:58,293 DEBUG [RS:1;jenkins-hbase12:35657] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:43555 2023-02-07 15:53:58,293 DEBUG [RS:1;jenkins-hbase12:35657] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-07 15:53:58,293 DEBUG [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:43555 2023-02-07 15:53:58,294 DEBUG [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8 2023-02-07 15:53:58,294 DEBUG [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-07 15:53:58,294 DEBUG [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:43555 2023-02-07 15:53:58,295 DEBUG [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-07 15:53:58,305 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-07 15:53:58,320 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-07 15:53:58,322 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-07 15:53:58,336 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-07 15:53:58,337 DEBUG [RS:1;jenkins-hbase12:35657] zookeeper.ZKUtil(162): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:53:58,337 DEBUG [RS:0;jenkins-hbase12:38629] zookeeper.ZKUtil(162): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:53:58,337 DEBUG [RS:2;jenkins-hbase12:37399] zookeeper.ZKUtil(162): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:53:58,337 WARN [RS:1;jenkins-hbase12:35657] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-07 15:53:58,338 WARN [RS:2;jenkins-hbase12:37399] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-07 15:53:58,337 WARN [RS:0;jenkins-hbase12:38629] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-07 15:53:58,338 INFO [RS:2;jenkins-hbase12:37399] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-07 15:53:58,338 INFO [RS:1;jenkins-hbase12:35657] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-07 15:53:58,339 INFO [RS:0;jenkins-hbase12:38629] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-07 15:53:58,339 DEBUG [RS:1;jenkins-hbase12:35657] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/WALs/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:53:58,339 DEBUG [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/WALs/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:53:58,339 DEBUG [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/WALs/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:53:58,341 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase12.apache.org,38629,1675785236797] 2023-02-07 15:53:58,342 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase12.apache.org,37399,1675785236898] 2023-02-07 15:53:58,342 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase12.apache.org,35657,1675785236854] 2023-02-07 15:53:58,352 DEBUG [RS:2;jenkins-hbase12:37399] zookeeper.ZKUtil(162): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:53:58,352 DEBUG [RS:0;jenkins-hbase12:38629] zookeeper.ZKUtil(162): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:53:58,352 DEBUG [RS:1;jenkins-hbase12:35657] zookeeper.ZKUtil(162): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:53:58,352 DEBUG [RS:2;jenkins-hbase12:37399] zookeeper.ZKUtil(162): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:53:58,352 DEBUG [RS:0;jenkins-hbase12:38629] zookeeper.ZKUtil(162): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:53:58,352 DEBUG [RS:1;jenkins-hbase12:35657] zookeeper.ZKUtil(162): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:53:58,353 DEBUG [RS:2;jenkins-hbase12:37399] zookeeper.ZKUtil(162): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:53:58,353 DEBUG [RS:0;jenkins-hbase12:38629] zookeeper.ZKUtil(162): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:53:58,353 DEBUG [RS:1;jenkins-hbase12:35657] zookeeper.ZKUtil(162): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:53:58,362 DEBUG [RS:0;jenkins-hbase12:38629] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-07 15:53:58,362 DEBUG [RS:2;jenkins-hbase12:37399] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-07 15:53:58,362 DEBUG [RS:1;jenkins-hbase12:35657] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-07 15:53:58,371 INFO [RS:0;jenkins-hbase12:38629] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-07 15:53:58,371 INFO [RS:2;jenkins-hbase12:37399] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-07 15:53:58,371 INFO [RS:1;jenkins-hbase12:35657] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-07 15:53:58,390 INFO [RS:0;jenkins-hbase12:38629] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-07 15:53:58,390 INFO [RS:1;jenkins-hbase12:35657] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-07 15:53:58,390 INFO [RS:2;jenkins-hbase12:37399] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-07 15:53:58,394 INFO [RS:0;jenkins-hbase12:38629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-07 15:53:58,394 INFO [RS:1;jenkins-hbase12:35657] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-07 15:53:58,395 INFO [RS:0;jenkins-hbase12:38629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,396 INFO [RS:2;jenkins-hbase12:37399] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-07 15:53:58,395 INFO [RS:1;jenkins-hbase12:35657] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,396 INFO [RS:2;jenkins-hbase12:37399] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,397 INFO [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-07 15:53:58,397 INFO [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-07 15:53:58,397 INFO [RS:1;jenkins-hbase12:35657] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-07 15:53:58,405 INFO [RS:0;jenkins-hbase12:38629] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,405 INFO [RS:2;jenkins-hbase12:37399] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,405 INFO [RS:1;jenkins-hbase12:35657] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,405 DEBUG [RS:0;jenkins-hbase12:38629] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,405 DEBUG [RS:2;jenkins-hbase12:37399] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,406 DEBUG [RS:1;jenkins-hbase12:35657] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,406 DEBUG [RS:2;jenkins-hbase12:37399] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,406 DEBUG [RS:1;jenkins-hbase12:35657] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,406 DEBUG [RS:0;jenkins-hbase12:38629] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,406 DEBUG [RS:1;jenkins-hbase12:35657] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,406 DEBUG [RS:0;jenkins-hbase12:38629] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,406 DEBUG [RS:1;jenkins-hbase12:35657] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,406 DEBUG [RS:2;jenkins-hbase12:37399] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,406 DEBUG [RS:1;jenkins-hbase12:35657] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,406 DEBUG [RS:2;jenkins-hbase12:37399] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,406 DEBUG [RS:0;jenkins-hbase12:38629] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:2;jenkins-hbase12:37399] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,406 DEBUG [RS:1;jenkins-hbase12:35657] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-02-07 15:53:58,407 DEBUG [RS:2;jenkins-hbase12:37399] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-02-07 15:53:58,407 DEBUG [RS:1;jenkins-hbase12:35657] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:0;jenkins-hbase12:38629] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:2;jenkins-hbase12:37399] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:0;jenkins-hbase12:38629] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-02-07 15:53:58,407 DEBUG [RS:1;jenkins-hbase12:35657] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:0;jenkins-hbase12:38629] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:2;jenkins-hbase12:37399] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:0;jenkins-hbase12:38629] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:1;jenkins-hbase12:35657] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:0;jenkins-hbase12:38629] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:2;jenkins-hbase12:37399] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:0;jenkins-hbase12:38629] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:1;jenkins-hbase12:35657] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,407 DEBUG [RS:2;jenkins-hbase12:37399] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 15:53:58,411 INFO [RS:0;jenkins-hbase12:38629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,411 INFO [RS:2;jenkins-hbase12:37399] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,412 INFO [RS:0;jenkins-hbase12:38629] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,412 INFO [RS:2;jenkins-hbase12:37399] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,412 INFO [RS:1;jenkins-hbase12:35657] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,412 INFO [RS:2;jenkins-hbase12:37399] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,412 INFO [RS:0;jenkins-hbase12:38629] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,412 INFO [RS:1;jenkins-hbase12:35657] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,412 INFO [RS:1;jenkins-hbase12:35657] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,426 INFO [RS:1;jenkins-hbase12:35657] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-07 15:53:58,427 INFO [RS:2;jenkins-hbase12:37399] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-07 15:53:58,427 INFO [RS:0;jenkins-hbase12:38629] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-07 15:53:58,430 INFO [RS:1;jenkins-hbase12:35657] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,35657,1675785236854-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,430 INFO [RS:0;jenkins-hbase12:38629] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,38629,1675785236797-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,430 INFO [RS:2;jenkins-hbase12:37399] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,37399,1675785236898-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:58,445 INFO [RS:2;jenkins-hbase12:37399] regionserver.Replication(203): jenkins-hbase12.apache.org,37399,1675785236898 started 2023-02-07 15:53:58,445 INFO [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(1638): Serving as jenkins-hbase12.apache.org,37399,1675785236898, RpcServer on jenkins-hbase12.apache.org/136.243.104.168:37399, sessionid=0x1013e2134d40003 2023-02-07 15:53:58,445 DEBUG [RS:2;jenkins-hbase12:37399] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-07 15:53:58,445 DEBUG [RS:2;jenkins-hbase12:37399] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:53:58,445 DEBUG [RS:2;jenkins-hbase12:37399] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,37399,1675785236898' 2023-02-07 15:53:58,445 DEBUG [RS:2;jenkins-hbase12:37399] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:53:58,446 DEBUG [RS:2;jenkins-hbase12:37399] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:53:58,446 DEBUG [RS:2;jenkins-hbase12:37399] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-07 15:53:58,446 DEBUG [RS:2;jenkins-hbase12:37399] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-07 15:53:58,447 DEBUG [RS:2;jenkins-hbase12:37399] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:53:58,447 DEBUG [RS:2;jenkins-hbase12:37399] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,37399,1675785236898' 2023-02-07 15:53:58,447 DEBUG [RS:2;jenkins-hbase12:37399] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-07 15:53:58,447 DEBUG [RS:2;jenkins-hbase12:37399] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-07 15:53:58,447 DEBUG [RS:2;jenkins-hbase12:37399] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-07 15:53:58,448 INFO [RS:2;jenkins-hbase12:37399] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-07 15:53:58,448 INFO [RS:2;jenkins-hbase12:37399] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-07 15:53:58,453 INFO [RS:1;jenkins-hbase12:35657] regionserver.Replication(203): jenkins-hbase12.apache.org,35657,1675785236854 started 2023-02-07 15:53:58,453 INFO [RS:0;jenkins-hbase12:38629] regionserver.Replication(203): jenkins-hbase12.apache.org,38629,1675785236797 started 2023-02-07 15:53:58,453 INFO [RS:1;jenkins-hbase12:35657] regionserver.HRegionServer(1638): Serving as jenkins-hbase12.apache.org,35657,1675785236854, RpcServer on jenkins-hbase12.apache.org/136.243.104.168:35657, sessionid=0x1013e2134d40002 2023-02-07 15:53:58,453 INFO [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(1638): Serving as jenkins-hbase12.apache.org,38629,1675785236797, RpcServer on jenkins-hbase12.apache.org/136.243.104.168:38629, sessionid=0x1013e2134d40001 2023-02-07 15:53:58,453 DEBUG [RS:1;jenkins-hbase12:35657] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-07 15:53:58,453 DEBUG [RS:0;jenkins-hbase12:38629] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-07 15:53:58,454 DEBUG [RS:0;jenkins-hbase12:38629] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:53:58,453 DEBUG [RS:1;jenkins-hbase12:35657] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:53:58,454 DEBUG [RS:1;jenkins-hbase12:35657] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,35657,1675785236854' 2023-02-07 15:53:58,454 DEBUG [RS:1;jenkins-hbase12:35657] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:53:58,454 DEBUG [RS:0;jenkins-hbase12:38629] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,38629,1675785236797' 2023-02-07 15:53:58,454 DEBUG [RS:0;jenkins-hbase12:38629] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:53:58,455 DEBUG [RS:0;jenkins-hbase12:38629] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:53:58,455 DEBUG [RS:1;jenkins-hbase12:35657] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:53:58,455 DEBUG [RS:0;jenkins-hbase12:38629] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-07 15:53:58,455 DEBUG [RS:1;jenkins-hbase12:35657] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-07 15:53:58,455 DEBUG [RS:0;jenkins-hbase12:38629] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-07 15:53:58,456 DEBUG [RS:1;jenkins-hbase12:35657] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-07 15:53:58,456 DEBUG [RS:0;jenkins-hbase12:38629] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:53:58,456 DEBUG [RS:1;jenkins-hbase12:35657] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:53:58,456 DEBUG [RS:0;jenkins-hbase12:38629] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,38629,1675785236797' 2023-02-07 15:53:58,456 DEBUG [RS:0;jenkins-hbase12:38629] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-07 15:53:58,456 DEBUG [RS:1;jenkins-hbase12:35657] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,35657,1675785236854' 2023-02-07 15:53:58,457 DEBUG [RS:1;jenkins-hbase12:35657] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-07 15:53:58,457 DEBUG [RS:0;jenkins-hbase12:38629] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-07 15:53:58,457 DEBUG [RS:1;jenkins-hbase12:35657] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-07 15:53:58,457 DEBUG [RS:0;jenkins-hbase12:38629] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-07 15:53:58,457 INFO [RS:0;jenkins-hbase12:38629] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-07 15:53:58,458 DEBUG [RS:1;jenkins-hbase12:35657] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-07 15:53:58,458 INFO [RS:0;jenkins-hbase12:38629] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-07 15:53:58,459 INFO [RS:1;jenkins-hbase12:35657] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-07 15:53:58,459 INFO [RS:1;jenkins-hbase12:35657] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-07 15:53:58,474 DEBUG [jenkins-hbase12:37073] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-07 15:53:58,477 DEBUG [jenkins-hbase12:37073] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:53:58,482 DEBUG [jenkins-hbase12:37073] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:53:58,482 DEBUG [jenkins-hbase12:37073] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:53:58,482 DEBUG [jenkins-hbase12:37073] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:53:58,482 DEBUG [jenkins-hbase12:37073] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 15:53:58,485 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase12.apache.org,37399,1675785236898, state=OPENING 2023-02-07 15:53:58,508 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-07 15:53:58,518 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 15:53:58,519 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-07 15:53:58,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase12.apache.org,37399,1675785236898}] 2023-02-07 15:53:58,557 INFO [RS:2;jenkins-hbase12:37399] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C37399%2C1675785236898, suffix=, logDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/WALs/jenkins-hbase12.apache.org,37399,1675785236898, archiveDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/oldWALs, maxLogs=32 2023-02-07 15:53:58,562 INFO [RS:1;jenkins-hbase12:35657] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C35657%2C1675785236854, suffix=, logDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/WALs/jenkins-hbase12.apache.org,35657,1675785236854, archiveDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/oldWALs, maxLogs=32 2023-02-07 15:53:58,562 INFO [RS:0;jenkins-hbase12:38629] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C38629%2C1675785236797, suffix=, logDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/WALs/jenkins-hbase12.apache.org,38629,1675785236797, archiveDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/oldWALs, maxLogs=32 2023-02-07 15:53:58,579 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41793,DS-0bc155ad-6b7d-4aab-8f0f-157a0fc146e9,DISK] 2023-02-07 15:53:58,579 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41397,DS-4c327e91-f723-45ab-baa6-5e10ff9a92ea,DISK] 2023-02-07 15:53:58,579 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41359,DS-f1e4096b-8cd7-4760-a5b7-364d59fe273e,DISK] 2023-02-07 15:53:58,589 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41793,DS-0bc155ad-6b7d-4aab-8f0f-157a0fc146e9,DISK] 2023-02-07 15:53:58,589 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41397,DS-4c327e91-f723-45ab-baa6-5e10ff9a92ea,DISK] 2023-02-07 15:53:58,590 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41359,DS-f1e4096b-8cd7-4760-a5b7-364d59fe273e,DISK] 2023-02-07 15:53:58,599 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41359,DS-f1e4096b-8cd7-4760-a5b7-364d59fe273e,DISK] 2023-02-07 15:53:58,602 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41397,DS-4c327e91-f723-45ab-baa6-5e10ff9a92ea,DISK] 2023-02-07 15:53:58,602 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41793,DS-0bc155ad-6b7d-4aab-8f0f-157a0fc146e9,DISK] 2023-02-07 15:53:58,604 INFO [RS:2;jenkins-hbase12:37399] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/WALs/jenkins-hbase12.apache.org,37399,1675785236898/jenkins-hbase12.apache.org%2C37399%2C1675785236898.1675785238559 2023-02-07 15:53:58,604 DEBUG [RS:2;jenkins-hbase12:37399] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41359,DS-f1e4096b-8cd7-4760-a5b7-364d59fe273e,DISK], DatanodeInfoWithStorage[127.0.0.1:41793,DS-0bc155ad-6b7d-4aab-8f0f-157a0fc146e9,DISK], DatanodeInfoWithStorage[127.0.0.1:41397,DS-4c327e91-f723-45ab-baa6-5e10ff9a92ea,DISK]] 2023-02-07 15:53:58,613 INFO [RS:0;jenkins-hbase12:38629] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/WALs/jenkins-hbase12.apache.org,38629,1675785236797/jenkins-hbase12.apache.org%2C38629%2C1675785236797.1675785238564 2023-02-07 15:53:58,613 INFO [RS:1;jenkins-hbase12:35657] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/WALs/jenkins-hbase12.apache.org,35657,1675785236854/jenkins-hbase12.apache.org%2C35657%2C1675785236854.1675785238564 2023-02-07 15:53:58,614 DEBUG [RS:0;jenkins-hbase12:38629] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41359,DS-f1e4096b-8cd7-4760-a5b7-364d59fe273e,DISK], DatanodeInfoWithStorage[127.0.0.1:41397,DS-4c327e91-f723-45ab-baa6-5e10ff9a92ea,DISK], DatanodeInfoWithStorage[127.0.0.1:41793,DS-0bc155ad-6b7d-4aab-8f0f-157a0fc146e9,DISK]] 2023-02-07 15:53:58,614 DEBUG [RS:1;jenkins-hbase12:35657] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41793,DS-0bc155ad-6b7d-4aab-8f0f-157a0fc146e9,DISK], DatanodeInfoWithStorage[127.0.0.1:41359,DS-f1e4096b-8cd7-4760-a5b7-364d59fe273e,DISK], DatanodeInfoWithStorage[127.0.0.1:41397,DS-4c327e91-f723-45ab-baa6-5e10ff9a92ea,DISK]] 2023-02-07 15:53:58,702 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:53:58,704 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-07 15:53:58,707 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:39808, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-07 15:53:58,722 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-07 15:53:58,723 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-07 15:53:58,727 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C37399%2C1675785236898.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/WALs/jenkins-hbase12.apache.org,37399,1675785236898, archiveDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/oldWALs, maxLogs=32 2023-02-07 15:53:58,749 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41793,DS-0bc155ad-6b7d-4aab-8f0f-157a0fc146e9,DISK] 2023-02-07 15:53:58,750 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41359,DS-f1e4096b-8cd7-4760-a5b7-364d59fe273e,DISK] 2023-02-07 15:53:58,752 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41397,DS-4c327e91-f723-45ab-baa6-5e10ff9a92ea,DISK] 2023-02-07 15:53:58,756 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/WALs/jenkins-hbase12.apache.org,37399,1675785236898/jenkins-hbase12.apache.org%2C37399%2C1675785236898.meta.1675785238728.meta 2023-02-07 15:53:58,757 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41793,DS-0bc155ad-6b7d-4aab-8f0f-157a0fc146e9,DISK], DatanodeInfoWithStorage[127.0.0.1:41359,DS-f1e4096b-8cd7-4760-a5b7-364d59fe273e,DISK], DatanodeInfoWithStorage[127.0.0.1:41397,DS-4c327e91-f723-45ab-baa6-5e10ff9a92ea,DISK]] 2023-02-07 15:53:58,758 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:53:58,762 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:53:58,763 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-07 15:53:58,778 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-07 15:53:58,781 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-07 15:53:58,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-07 15:53:58,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:53:58,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-07 15:53:58,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-07 15:53:58,788 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-07 15:53:58,790 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/info 2023-02-07 15:53:58,790 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/info 2023-02-07 15:53:58,790 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-07 15:53:58,791 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:53:58,791 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-07 15:53:58,793 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/rep_barrier 2023-02-07 15:53:58,793 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/rep_barrier 2023-02-07 15:53:58,794 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-07 15:53:58,794 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:53:58,794 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-07 15:53:58,796 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/table 2023-02-07 15:53:58,796 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/table 2023-02-07 15:53:58,797 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-07 15:53:58,797 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:53:58,799 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740 2023-02-07 15:53:58,803 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740 2023-02-07 15:53:58,806 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-07 15:53:58,808 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-07 15:53:58,809 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67227194, jitterRate=0.0017632544040679932}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-07 15:53:58,810 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675785238785Writing region info on filesystem at 1675785238786 (+1 ms)Initializing all the Stores at 1675785238787 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675785238788 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785238788Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675785238788Cleaning up temporary data from old regions at 1675785238804 (+16 ms)Cleaning up detritus from prior splits at 1675785238805 (+1 ms)Running coprocessor post-open hooks at 1675785238809 (+4 ms)Region opened successfully at 1675785238810 (+1 ms) 2023-02-07 15:53:58,819 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675785238695 2023-02-07 15:53:58,838 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-07 15:53:58,839 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-07 15:53:58,839 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase12.apache.org,37399,1675785236898, state=OPEN 2023-02-07 15:53:58,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-07 15:53:58,851 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-07 15:53:58,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-07 15:53:58,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase12.apache.org,37399,1675785236898 in 328 msec 2023-02-07 15:53:58,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-07 15:53:58,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 553 msec 2023-02-07 15:53:58,869 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 819 msec 2023-02-07 15:53:58,869 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675785236960, completionTime=-1 2023-02-07 15:53:58,869 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-07 15:53:58,869 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:53:58,923 DEBUG [hconnection-0xa17a051-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-07 15:53:58,927 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:39824, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-07 15:53:58,945 INFO [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:53:58,946 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785298946 2023-02-07 15:53:58,946 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785358946 2023-02-07 15:53:58,946 INFO [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 76 msec 2023-02-07 15:53:59,032 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,37073,1675785235583-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:59,032 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,37073,1675785235583-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:59,032 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,37073,1675785235583-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:59,034 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase12:37073, period=300000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:59,034 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-07 15:53:59,046 DEBUG [master/jenkins-hbase12:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-07 15:53:59,054 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-07 15:53:59,056 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-07 15:53:59,066 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-07 15:53:59,069 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 15:53:59,072 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 15:53:59,095 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6 2023-02-07 15:53:59,098 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6 empty. 2023-02-07 15:53:59,099 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6 2023-02-07 15:53:59,099 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-07 15:53:59,134 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-07 15:53:59,136 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2b3e2412ef7ea40bb26319d571a3f2e6, NAME => 'hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp 2023-02-07 15:53:59,157 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:53:59,158 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 2b3e2412ef7ea40bb26319d571a3f2e6, disabling compactions & flushes 2023-02-07 15:53:59,158 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. 2023-02-07 15:53:59,158 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. 2023-02-07 15:53:59,158 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. after waiting 0 ms 2023-02-07 15:53:59,158 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. 2023-02-07 15:53:59,158 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. 2023-02-07 15:53:59,158 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 2b3e2412ef7ea40bb26319d571a3f2e6: Waiting for close lock at 1675785239157Disabling compacts and flushes for region at 1675785239157Disabling writes for close at 1675785239158 (+1 ms)Writing region close event to WAL at 1675785239158Closed at 1675785239158 2023-02-07 15:53:59,163 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 15:53:59,177 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675785239166"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785239166"}]},"ts":"1675785239166"} 2023-02-07 15:53:59,206 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 15:53:59,208 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 15:53:59,213 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785239208"}]},"ts":"1675785239208"} 2023-02-07 15:53:59,219 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-07 15:53:59,241 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:53:59,242 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:53:59,242 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:53:59,242 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:53:59,242 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 15:53:59,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=2b3e2412ef7ea40bb26319d571a3f2e6, ASSIGN}] 2023-02-07 15:53:59,248 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=2b3e2412ef7ea40bb26319d571a3f2e6, ASSIGN 2023-02-07 15:53:59,251 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=2b3e2412ef7ea40bb26319d571a3f2e6, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:53:59,403 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:53:59,404 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:53:59,405 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675785239404"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785239404"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785239404"}]},"ts":"1675785239404"} 2023-02-07 15:53:59,410 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 2b3e2412ef7ea40bb26319d571a3f2e6, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:53:59,564 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:53:59,565 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-07 15:53:59,568 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:57292, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-07 15:53:59,573 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. 2023-02-07 15:53:59,574 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2b3e2412ef7ea40bb26319d571a3f2e6, NAME => 'hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:53:59,575 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:53:59,576 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 2b3e2412ef7ea40bb26319d571a3f2e6 2023-02-07 15:53:59,576 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:53:59,576 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 2b3e2412ef7ea40bb26319d571a3f2e6 2023-02-07 15:53:59,576 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 2b3e2412ef7ea40bb26319d571a3f2e6 2023-02-07 15:53:59,578 INFO [StoreOpener-2b3e2412ef7ea40bb26319d571a3f2e6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 2b3e2412ef7ea40bb26319d571a3f2e6 2023-02-07 15:53:59,580 DEBUG [StoreOpener-2b3e2412ef7ea40bb26319d571a3f2e6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6/info 2023-02-07 15:53:59,580 DEBUG [StoreOpener-2b3e2412ef7ea40bb26319d571a3f2e6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6/info 2023-02-07 15:53:59,581 INFO [StoreOpener-2b3e2412ef7ea40bb26319d571a3f2e6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2b3e2412ef7ea40bb26319d571a3f2e6 columnFamilyName info 2023-02-07 15:53:59,582 INFO [StoreOpener-2b3e2412ef7ea40bb26319d571a3f2e6-1] regionserver.HStore(310): Store=2b3e2412ef7ea40bb26319d571a3f2e6/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:53:59,584 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6 2023-02-07 15:53:59,586 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6 2023-02-07 15:53:59,590 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 2b3e2412ef7ea40bb26319d571a3f2e6 2023-02-07 15:53:59,593 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:53:59,594 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 2b3e2412ef7ea40bb26319d571a3f2e6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73570740, jitterRate=0.09628945589065552}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-07 15:53:59,594 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 2b3e2412ef7ea40bb26319d571a3f2e6: Running coprocessor pre-open hook at 1675785239576Writing region info on filesystem at 1675785239576Initializing all the Stores at 1675785239578 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675785239578Cleaning up temporary data from old regions at 1675785239586 (+8 ms)Cleaning up detritus from prior splits at 1675785239587 (+1 ms)Running coprocessor post-open hooks at 1675785239594 (+7 ms)Region opened successfully at 1675785239594 2023-02-07 15:53:59,596 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6., pid=6, masterSystemTime=1675785239564 2023-02-07 15:53:59,601 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. 2023-02-07 15:53:59,602 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. 2023-02-07 15:53:59,604 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:53:59,605 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675785239603"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785239603"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785239603"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785239603"}]},"ts":"1675785239603"} 2023-02-07 15:53:59,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-07 15:53:59,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 2b3e2412ef7ea40bb26319d571a3f2e6, server=jenkins-hbase12.apache.org,38629,1675785236797 in 199 msec 2023-02-07 15:53:59,619 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-07 15:53:59,619 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=2b3e2412ef7ea40bb26319d571a3f2e6, ASSIGN in 370 msec 2023-02-07 15:53:59,621 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 15:53:59,622 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785239621"}]},"ts":"1675785239621"} 2023-02-07 15:53:59,624 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-07 15:53:59,636 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 15:53:59,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 580 msec 2023-02-07 15:53:59,669 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-07 15:53:59,682 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-07 15:53:59,682 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 15:53:59,703 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-07 15:53:59,706 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:57300, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-07 15:53:59,718 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-07 15:53:59,750 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-07 15:53:59,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 53 msec 2023-02-07 15:53:59,772 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:53:59,792 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-07 15:53:59,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 33 msec 2023-02-07 15:53:59,834 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-07 15:53:59,892 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-07 15:53:59,893 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.904sec 2023-02-07 15:53:59,895 INFO [master/jenkins-hbase12:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-07 15:53:59,899 INFO [master/jenkins-hbase12:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-07 15:53:59,900 INFO [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-07 15:53:59,901 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,37073,1675785235583-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-07 15:53:59,903 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,37073,1675785235583-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-07 15:53:59,911 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-07 15:53:59,937 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ReadOnlyZKClient(139): Connect 0x7900af31 to 127.0.0.1:51095 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-07 15:53:59,951 DEBUG [Listener at localhost.localdomain/41965] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3d358578, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-07 15:53:59,987 DEBUG [hconnection-0x6ba35714-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-07 15:54:00,006 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:39832, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-07 15:54:00,014 INFO [Listener at localhost.localdomain/41965] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase12.apache.org,37073,1675785235583 2023-02-07 15:54:00,040 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=362, OpenFileDescriptor=700, MaxFileDescriptor=60000, SystemLoadAverage=383, ProcessCount=173, AvailableMemoryMB=3373 2023-02-07 15:54:00,044 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:00,044 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:00,045 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:00,051 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-07 15:54:00,054 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:58182, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-07 15:54:00,086 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-07 15:54:00,087 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 15:54:00,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 15:54:00,093 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,095 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 15:54:00,097 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 15:54:00,100 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:00,101 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028 empty. 2023-02-07 15:54:00,103 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:00,103 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-07 15:54:00,128 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-07 15:54:00,130 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1e613dcf57a1f9c06070a32173e39028, NAME => 'testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp 2023-02-07 15:54:00,150 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:00,150 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing 1e613dcf57a1f9c06070a32173e39028, disabling compactions & flushes 2023-02-07 15:54:00,150 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:00,150 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:00,150 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. after waiting 0 ms 2023-02-07 15:54:00,150 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:00,150 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:00,150 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 1e613dcf57a1f9c06070a32173e39028: Waiting for close lock at 1675785240150Disabling compacts and flushes for region at 1675785240150Disabling writes for close at 1675785240150Writing region close event to WAL at 1675785240150Closed at 1675785240150 2023-02-07 15:54:00,154 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 15:54:00,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675785240156"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785240156"}]},"ts":"1675785240156"} 2023-02-07 15:54:00,159 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 15:54:00,161 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 15:54:00,161 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785240161"}]},"ts":"1675785240161"} 2023-02-07 15:54:00,163 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-07 15:54:00,188 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:54:00,190 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:54:00,190 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:54:00,190 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:54:00,190 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 15:54:00,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1e613dcf57a1f9c06070a32173e39028, ASSIGN}] 2023-02-07 15:54:00,194 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1e613dcf57a1f9c06070a32173e39028, ASSIGN 2023-02-07 15:54:00,195 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1e613dcf57a1f9c06070a32173e39028, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,35657,1675785236854; forceNewPlan=false, retain=false 2023-02-07 15:54:00,346 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:54:00,347 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=1e613dcf57a1f9c06070a32173e39028, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:00,347 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675785240346"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785240346"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785240346"}]},"ts":"1675785240346"} 2023-02-07 15:54:00,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 1e613dcf57a1f9c06070a32173e39028, server=jenkins-hbase12.apache.org,35657,1675785236854}] 2023-02-07 15:54:00,504 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:00,504 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-07 15:54:00,507 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:60648, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-07 15:54:00,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:00,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1e613dcf57a1f9c06070a32173e39028, NAME => 'testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:54:00,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:00,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:00,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:00,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:00,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:00,516 INFO [StoreOpener-1e613dcf57a1f9c06070a32173e39028-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:00,518 DEBUG [StoreOpener-1e613dcf57a1f9c06070a32173e39028-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1 2023-02-07 15:54:00,518 DEBUG [StoreOpener-1e613dcf57a1f9c06070a32173e39028-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1 2023-02-07 15:54:00,519 INFO [StoreOpener-1e613dcf57a1f9c06070a32173e39028-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1e613dcf57a1f9c06070a32173e39028 columnFamilyName cf1 2023-02-07 15:54:00,520 INFO [StoreOpener-1e613dcf57a1f9c06070a32173e39028-1] regionserver.HStore(310): Store=1e613dcf57a1f9c06070a32173e39028/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:00,520 INFO [StoreOpener-1e613dcf57a1f9c06070a32173e39028-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:00,522 DEBUG [StoreOpener-1e613dcf57a1f9c06070a32173e39028-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2 2023-02-07 15:54:00,522 DEBUG [StoreOpener-1e613dcf57a1f9c06070a32173e39028-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2 2023-02-07 15:54:00,523 INFO [StoreOpener-1e613dcf57a1f9c06070a32173e39028-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1e613dcf57a1f9c06070a32173e39028 columnFamilyName cf2 2023-02-07 15:54:00,524 INFO [StoreOpener-1e613dcf57a1f9c06070a32173e39028-1] regionserver.HStore(310): Store=1e613dcf57a1f9c06070a32173e39028/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:00,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:00,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:00,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:00,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:00,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:54:00,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 1e613dcf57a1f9c06070a32173e39028; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67522250, jitterRate=0.006159931421279907}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:00,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 1e613dcf57a1f9c06070a32173e39028: Running coprocessor pre-open hook at 1675785240514Writing region info on filesystem at 1675785240514Initializing all the Stores at 1675785240515 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785240515Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785240515Cleaning up temporary data from old regions at 1675785240528 (+13 ms)Cleaning up detritus from prior splits at 1675785240529 (+1 ms)Running coprocessor post-open hooks at 1675785240537 (+8 ms)Region opened successfully at 1675785240537 2023-02-07 15:54:00,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028., pid=11, masterSystemTime=1675785240504 2023-02-07 15:54:00,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:00,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:00,545 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=1e613dcf57a1f9c06070a32173e39028, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:00,546 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675785240544"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785240544"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785240544"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785240544"}]},"ts":"1675785240544"} 2023-02-07 15:54:00,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-07 15:54:00,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 1e613dcf57a1f9c06070a32173e39028, server=jenkins-hbase12.apache.org,35657,1675785236854 in 199 msec 2023-02-07 15:54:00,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-07 15:54:00,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1e613dcf57a1f9c06070a32173e39028, ASSIGN in 363 msec 2023-02-07 15:54:00,558 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 15:54:00,558 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785240558"}]},"ts":"1675785240558"} 2023-02-07 15:54:00,561 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-07 15:54:00,614 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 15:54:00,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 523 msec 2023-02-07 15:54:00,848 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-07 15:54:00,853 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:60662, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-07 15:54:00,870 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:00,870 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:00,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:00,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,881 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:00,881 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:00,882 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-07 15:54:00,882 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-07 15:54:00,883 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,883 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,966 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:00,966 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:00,966 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:00,966 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:00,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:00,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:00,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:00,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:00,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:00,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:00,967 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:00,967 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:00,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:00,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:00,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:00,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,968 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,968 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,968 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,970 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:00,970 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:00,970 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:00,970 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:00,970 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:00,971 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:00,972 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:00,970 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:00,972 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:00,972 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:00,972 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:00,972 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:00,972 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:00,973 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:00,973 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:00,973 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:00,973 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. started... 2023-02-07 15:54:00,975 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e613dcf57a1f9c06070a32173e39028 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 15:54:00,981 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,991 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,991 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,992 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:00,992 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:00,992 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:01,043 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/5ba564e3bd7e4a38bf3f64187c686c6a 2023-02-07 15:54:01,103 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/74d0eb3700ef440d9a093976ba929bfa 2023-02-07 15:54:01,114 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/5ba564e3bd7e4a38bf3f64187c686c6a as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/5ba564e3bd7e4a38bf3f64187c686c6a 2023-02-07 15:54:01,125 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/5ba564e3bd7e4a38bf3f64187c686c6a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:54:01,128 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/74d0eb3700ef440d9a093976ba929bfa as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/74d0eb3700ef440d9a093976ba929bfa 2023-02-07 15:54:01,138 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/74d0eb3700ef440d9a093976ba929bfa, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:54:01,140 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 1e613dcf57a1f9c06070a32173e39028 in 165ms, sequenceid=6, compaction requested=false 2023-02-07 15:54:01,142 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:01,144 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e613dcf57a1f9c06070a32173e39028: 2023-02-07 15:54:01,144 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:01,144 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:01,145 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:01,145 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,145 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:01,145 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:01,156 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,156 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:01,156 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,156 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:01,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:01,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:01,159 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-07 15:54:01,159 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:01,159 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@eb47b60[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:01,159 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,202 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,202 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,202 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,203 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:01,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,203 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:01,203 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,203 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:01,203 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:01,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,203 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:01,203 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:01,203 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:01,203 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:01,203 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:01,203 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:01,225 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:01,225 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,225 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:01,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:01,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:01,225 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:01,234 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:01,234 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:01,234 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:01,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:01,234 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:01,235 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:01,234 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:01,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:01,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:01,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,35657,1675785236854': 2023-02-07 15:54:01,240 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-07 15:54:01,240 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:01,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:01,240 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:01,240 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,240 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:01,250 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,250 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:01,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:01,250 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,250 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,250 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,251 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,251 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,251 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:01,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,252 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:01,252 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:01,255 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,255 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,255 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,271 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,271 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,272 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,272 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,271 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,272 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,272 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,272 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:01,272 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:01,272 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,272 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,273 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:01,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,273 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,273 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,274 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,274 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,274 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,274 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,275 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:01,275 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:01,375 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:01,376 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:01,376 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:01,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:01,408 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:01,408 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:01,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:01,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:01,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,412 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:01,412 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:01,413 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-07 15:54:01,413 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-07 15:54:01,414 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,414 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,423 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,424 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,424 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:01,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:01,426 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:01,426 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,426 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:01,426 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:01,426 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:01,426 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:01,427 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:01,426 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:01,427 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:01,427 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:01,427 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:01,427 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:01,427 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:01,427 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:01,427 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:01,427 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:01,427 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:01,427 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. started... 2023-02-07 15:54:01,428 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e613dcf57a1f9c06070a32173e39028 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:01,440 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,448 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/9e05977f40a2406d84575de48fd21a0b 2023-02-07 15:54:01,450 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,450 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,450 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:01,450 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,451 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:01,474 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/fb60bbc8741c4c9181107980b9b3bc10 2023-02-07 15:54:01,484 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/9e05977f40a2406d84575de48fd21a0b as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/9e05977f40a2406d84575de48fd21a0b 2023-02-07 15:54:01,494 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/9e05977f40a2406d84575de48fd21a0b, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:54:01,496 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/fb60bbc8741c4c9181107980b9b3bc10 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/fb60bbc8741c4c9181107980b9b3bc10 2023-02-07 15:54:01,506 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/fb60bbc8741c4c9181107980b9b3bc10, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:54:01,507 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1e613dcf57a1f9c06070a32173e39028 in 79ms, sequenceid=19, compaction requested=false 2023-02-07 15:54:01,507 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e613dcf57a1f9c06070a32173e39028: 2023-02-07 15:54:01,508 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:01,508 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:01,508 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:01,508 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,508 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:01,508 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:01,518 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,518 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:01,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:01,518 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,518 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:01,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:01,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:01,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:01,521 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-07 15:54:01,521 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2846c824[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:01,521 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:01,521 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,528 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,528 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,528 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,529 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,529 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:01,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,529 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:01,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,529 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:01,529 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:01,529 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:01,529 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:01,529 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:01,529 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:01,529 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:01,529 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:01,540 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:01,540 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:01,540 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:01,549 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:01,550 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:01,550 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:01,550 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,549 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:01,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:01,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:01,551 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:01,551 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:01,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:01,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:01,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:01,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,35657,1675785236854': 2023-02-07 15:54:01,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:01,555 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-07 15:54:01,555 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:01,555 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:01,555 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,555 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:01,566 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,566 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,566 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,566 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,566 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,566 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,566 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:01,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:01,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,567 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:01,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,568 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,568 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:01,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:01,570 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,570 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,570 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,613 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,613 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,613 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,614 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,614 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:01,614 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,614 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:01,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:01,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,614 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,615 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,615 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,616 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,616 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:01,616 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:01,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,616 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,616 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,616 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,616 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,616 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,616 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,616 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,716 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:01,716 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:01,716 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:01,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:01,743 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:01,743 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:01,744 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:01,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:01,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,746 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:01,746 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:01,747 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-07 15:54:01,747 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-07 15:54:01,747 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,747 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,766 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,766 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,766 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,766 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,767 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,767 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:01,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,768 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,768 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:01,768 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:01,768 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:01,768 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,769 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:01,769 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:01,768 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,769 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:01,769 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:01,769 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:01,769 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:01,770 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:01,770 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:01,771 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:01,771 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:01,771 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:01,771 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:01,771 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:01,771 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:01,772 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. started... 2023-02-07 15:54:01,772 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e613dcf57a1f9c06070a32173e39028 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:01,781 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,791 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,791 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/0cc2026840b04973a4109de67a8ebefd 2023-02-07 15:54:01,791 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,791 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:01,792 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,792 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:01,816 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/5dd80339a63f49cdae13cfe694bd4144 2023-02-07 15:54:01,826 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/0cc2026840b04973a4109de67a8ebefd as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/0cc2026840b04973a4109de67a8ebefd 2023-02-07 15:54:01,835 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/0cc2026840b04973a4109de67a8ebefd, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:54:01,836 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/5dd80339a63f49cdae13cfe694bd4144 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/5dd80339a63f49cdae13cfe694bd4144 2023-02-07 15:54:01,844 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/5dd80339a63f49cdae13cfe694bd4144, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:54:01,845 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1e613dcf57a1f9c06070a32173e39028 in 73ms, sequenceid=32, compaction requested=true 2023-02-07 15:54:01,846 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e613dcf57a1f9c06070a32173e39028: 2023-02-07 15:54:01,846 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:01,846 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:01,846 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:01,846 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,847 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:01,847 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:01,855 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,855 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:01,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:01,855 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,855 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:01,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:01,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:01,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:01,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-07 15:54:01,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@547242bc[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:01,858 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:01,858 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,865 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,865 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,865 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,866 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:01,866 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,866 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:01,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,866 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:01,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,866 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:01,866 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:01,866 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:01,866 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:01,866 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:01,866 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:01,866 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:01,878 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:01,878 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,878 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:01,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:01,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:01,878 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:01,886 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:01,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:01,886 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:01,887 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:01,886 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:01,888 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:01,888 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:01,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:01,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:01,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,35657,1675785236854': 2023-02-07 15:54:01,892 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:01,892 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-07 15:54:01,892 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:01,892 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:01,892 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,892 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:01,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:01,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,904 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:01,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,904 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:01,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,905 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,906 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,906 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,906 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:01,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:01,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:01,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,919 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,919 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:01,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:01,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:01,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:01,919 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:01,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:01,919 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,920 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,920 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:01,920 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,920 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:01,920 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:01,920 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:01,921 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,921 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:01,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,921 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:01,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:01,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:01,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter because node does not exist (not an error) 2023-02-07 15:54:02,021 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:02,021 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:02,021 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:02,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:02,049 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:02,049 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:02,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:02,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:02,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,052 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:02,053 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:02,053 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-07 15:54:02,053 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-07 15:54:02,054 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,054 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,065 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,065 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,065 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,065 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,066 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,066 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:02,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,067 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:02,067 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:02,067 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,067 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:02,067 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:02,067 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:02,067 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,068 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:02,068 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:02,068 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:02,068 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:02,068 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:02,069 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:02,069 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:02,069 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:02,069 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:02,069 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:02,070 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:02,070 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:02,070 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:02,070 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. started... 2023-02-07 15:54:02,070 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e613dcf57a1f9c06070a32173e39028 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:02,083 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,088 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/278f7a6eeae14592a3a39b2bb040a948 2023-02-07 15:54:02,092 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,092 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,092 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:02,093 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,093 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:02,111 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/812659b3292944e6a9024715a8a9a148 2023-02-07 15:54:02,120 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/278f7a6eeae14592a3a39b2bb040a948 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/278f7a6eeae14592a3a39b2bb040a948 2023-02-07 15:54:02,128 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/278f7a6eeae14592a3a39b2bb040a948, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:54:02,129 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/812659b3292944e6a9024715a8a9a148 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/812659b3292944e6a9024715a8a9a148 2023-02-07 15:54:02,138 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/812659b3292944e6a9024715a8a9a148, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:54:02,140 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1e613dcf57a1f9c06070a32173e39028 in 69ms, sequenceid=45, compaction requested=true 2023-02-07 15:54:02,140 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e613dcf57a1f9c06070a32173e39028: 2023-02-07 15:54:02,140 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:02,140 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:02,141 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:02,141 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,141 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:02,141 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:02,213 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,213 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:02,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:02,214 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,214 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:02,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:02,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:02,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:02,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-07 15:54:02,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25bc33a8[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:02,216 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:02,216 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,240 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,240 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,240 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,240 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,240 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:02,240 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:02,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,241 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:02,241 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:02,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,241 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:02,241 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:02,241 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:02,241 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:02,241 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:02,241 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:02,250 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:02,250 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:02,250 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:02,251 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:02,251 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:02,251 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:02,261 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:02,261 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:02,261 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:02,261 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:02,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:02,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:02,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:02,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:02,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,35657,1675785236854': 2023-02-07 15:54:02,266 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:02,266 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-07 15:54:02,266 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:02,266 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:02,266 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,266 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:02,276 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,276 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,276 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,276 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,276 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:02,276 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:02,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:02,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:02,276 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,276 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:02,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:02,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,277 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:02,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:02,277 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:02,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:02,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:02,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:02,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:02,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:02,279 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,279 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,279 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,292 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,292 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,293 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:02,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:02,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:02,293 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:02,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,293 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,292 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,292 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,294 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:02,294 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,294 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:02,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,294 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:02,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,294 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:02,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:02,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:02,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:02,294 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:02,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:02,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:02,295 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,295 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,295 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,295 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,295 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,295 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,295 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,394 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:02,395 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:02,395 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:02,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:02,421 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:02,421 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:02,422 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:02,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:02,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,425 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:02,425 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:02,426 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-07 15:54:02,426 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-07 15:54:02,427 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,427 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,434 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,434 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,434 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,434 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,434 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,434 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:02,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,434 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,435 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:02,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,435 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,435 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:02,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,435 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,435 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:02,435 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,435 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:02,435 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:02,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:02,438 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:02,435 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:02,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,438 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:02,438 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:02,438 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. started... 2023-02-07 15:54:02,438 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:02,438 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:02,438 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:02,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,438 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:02,438 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e613dcf57a1f9c06070a32173e39028 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:02,439 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:02,439 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:02,439 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:02,438 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:02,439 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:02,441 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:02,441 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:02,441 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:02,457 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,462 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/6c8f680c3c3f45d9a8e1e35445f4bbbe 2023-02-07 15:54:02,465 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,465 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,465 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:02,465 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,465 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:02,488 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/98775db67ca64b4e90613e8973fdbeb7 2023-02-07 15:54:02,495 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/6c8f680c3c3f45d9a8e1e35445f4bbbe as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/6c8f680c3c3f45d9a8e1e35445f4bbbe 2023-02-07 15:54:02,503 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/6c8f680c3c3f45d9a8e1e35445f4bbbe, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:54:02,504 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/98775db67ca64b4e90613e8973fdbeb7 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/98775db67ca64b4e90613e8973fdbeb7 2023-02-07 15:54:02,512 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/98775db67ca64b4e90613e8973fdbeb7, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:54:02,513 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1e613dcf57a1f9c06070a32173e39028 in 75ms, sequenceid=58, compaction requested=true 2023-02-07 15:54:02,513 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e613dcf57a1f9c06070a32173e39028: 2023-02-07 15:54:02,513 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:02,513 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:02,514 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:02,514 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,514 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:02,514 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:02,524 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,524 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:02,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:02,525 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,525 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:02,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:02,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:02,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:02,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-07 15:54:02,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8fe3304[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:02,527 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:02,527 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,539 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,539 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,539 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,539 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,539 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:02,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,540 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:02,540 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:02,540 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:02,540 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:02,540 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:02,540 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:02,540 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:02,540 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:02,540 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:02,549 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:02,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,549 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:02,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:02,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:02,550 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:02,550 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:02,549 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:02,551 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:02,560 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:02,560 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:02,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:02,560 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:02,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:02,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:02,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,35657,1675785236854': 2023-02-07 15:54:02,566 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:02,566 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-07 15:54:02,566 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:02,566 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:02,566 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,566 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:02,577 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,577 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,577 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,577 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,577 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:02,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:02,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:02,577 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:02,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,577 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:02,577 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:02,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:02,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:02,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,578 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:02,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:02,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:02,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:02,578 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:02,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:02,580 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,580 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,580 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,649 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,649 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,649 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,649 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:02,649 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:02,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,649 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:02,649 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:02,650 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:02,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:02,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:02,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,649 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:02,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:02,650 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:02,650 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,650 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,650 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,650 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,650 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,651 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,651 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,651 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,651 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:02,651 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:02,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:02,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:02,751 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:02,751 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:02,751 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:02,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:02,776 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:02,776 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:02,777 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:02,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:02,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,780 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:02,780 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:02,780 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-07 15:54:02,780 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-07 15:54:02,781 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,781 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,802 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,802 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,802 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,802 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:02,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:02,803 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,803 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:02,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:02,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,804 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:02,804 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,804 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 15:54:02,804 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:02,804 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,805 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:02,805 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:02,805 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:02,805 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:02,806 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:02,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,806 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. started... 2023-02-07 15:54:02,806 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:02,806 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:02,806 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:02,806 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:02,806 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 15:54:02,806 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:02,806 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 15:54:02,806 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:02,807 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:02,807 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e613dcf57a1f9c06070a32173e39028 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:02,821 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,823 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/d7a6d58e70ab457bbe8f6f47977d9c1e 2023-02-07 15:54:02,829 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,829 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,829 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:02,830 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,830 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:02,845 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/41a5bd1025fd4c93816f3f1d0b0e4e7c 2023-02-07 15:54:02,853 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/d7a6d58e70ab457bbe8f6f47977d9c1e as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/d7a6d58e70ab457bbe8f6f47977d9c1e 2023-02-07 15:54:02,861 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/d7a6d58e70ab457bbe8f6f47977d9c1e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:54:02,862 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/41a5bd1025fd4c93816f3f1d0b0e4e7c as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/41a5bd1025fd4c93816f3f1d0b0e4e7c 2023-02-07 15:54:02,870 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/41a5bd1025fd4c93816f3f1d0b0e4e7c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:54:02,871 INFO [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1e613dcf57a1f9c06070a32173e39028 in 64ms, sequenceid=71, compaction requested=true 2023-02-07 15:54:02,871 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e613dcf57a1f9c06070a32173e39028: 2023-02-07 15:54:02,871 DEBUG [rs(jenkins-hbase12.apache.org,35657,1675785236854)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:02,872 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:02,872 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:02,872 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,872 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 15:54:02,872 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 15:54:02,882 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,882 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:02,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:02,882 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,883 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:02,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:02,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:02,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:02,885 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-07 15:54:02,885 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16e7cdd7[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:02,885 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:02,885 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,897 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,897 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,897 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,897 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:02,897 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:02,897 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,898 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:02,898 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:02,898 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:02,897 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 15:54:02,898 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:02,898 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:02,898 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 15:54:02,898 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:02,907 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:02,907 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:02,907 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,907 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:02,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:02,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:02,918 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:02,918 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:02,918 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 15:54:02,918 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:02,919 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:02,919 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 15:54:02,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:02,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:02,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:02,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:02,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:02,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,35657,1675785236854': 2023-02-07 15:54:02,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:02,923 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-07 15:54:02,923 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:02,923 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:02,923 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-07 15:54:02,923 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:03,007 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,007 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:03,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:03,007 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,007 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,007 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:03,007 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:03,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,007 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:03,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:03,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,008 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:03,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:03,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:03,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:03,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:03,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:03,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:03,008 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:03,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:03,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,009 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:03,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:03,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:03,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:03,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:03,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,010 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:03,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:03,010 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:03,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:03,011 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:03,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:03,135 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:03,135 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:03,135 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:03,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-07 15:54:03,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:03,135 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:03,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:03,136 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:03,136 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:03,136 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:03,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:03,136 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:03,137 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:03,137 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:03,137 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:03,138 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:03,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:03,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:03,136 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:03,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:03,138 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:03,138 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:03,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:03,138 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:03,138 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:03,139 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:03,139 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,139 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 15:54:03,238 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:03,238 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:03,238 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:03,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:03,262 DEBUG [Listener at localhost.localdomain/41965] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:03,269 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:03,269 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:03,270 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:03,270 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:03,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1e613dcf57a1f9c06070a32173e39028, UNASSIGN}] 2023-02-07 15:54:03,280 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1e613dcf57a1f9c06070a32173e39028, UNASSIGN 2023-02-07 15:54:03,281 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=1e613dcf57a1f9c06070a32173e39028, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:03,281 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675785243281"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785243281"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785243281"}]},"ts":"1675785243281"} 2023-02-07 15:54:03,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 1e613dcf57a1f9c06070a32173e39028, server=jenkins-hbase12.apache.org,35657,1675785236854}] 2023-02-07 15:54:03,444 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 1e613dcf57a1f9c06070a32173e39028, disabling compactions & flushes 2023-02-07 15:54:03,444 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:03,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:03,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. after waiting 0 ms 2023-02-07 15:54:03,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:03,444 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 1e613dcf57a1f9c06070a32173e39028 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 15:54:03,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/039f958077d545baba1659650978773c 2023-02-07 15:54:03,480 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/31b897ea219d4f8c9f8f6298a47c32a2 2023-02-07 15:54:03,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf1/039f958077d545baba1659650978773c as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/039f958077d545baba1659650978773c 2023-02-07 15:54:03,496 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/039f958077d545baba1659650978773c, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:54:03,498 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.tmp/cf2/31b897ea219d4f8c9f8f6298a47c32a2 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/31b897ea219d4f8c9f8f6298a47c32a2 2023-02-07 15:54:03,507 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/31b897ea219d4f8c9f8f6298a47c32a2, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:54:03,508 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1e613dcf57a1f9c06070a32173e39028 in 64ms, sequenceid=82, compaction requested=true 2023-02-07 15:54:03,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 15:54:03,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:03,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. 2023-02-07 15:54:03,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 1e613dcf57a1f9c06070a32173e39028: Waiting for close lock at 1675785243444Running coprocessor pre-close hooks at 1675785243444Disabling compacts and flushes for region at 1675785243444Disabling writes for close at 1675785243444Obtaining lock to block concurrent updates at 1675785243444Preparing flush snapshotting stores in 1e613dcf57a1f9c06070a32173e39028 at 1675785243444Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675785243444Flushing stores of testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. at 1675785243445 (+1 ms)Flushing 1e613dcf57a1f9c06070a32173e39028/cf1: creating writer at 1675785243446 (+1 ms)Flushing 1e613dcf57a1f9c06070a32173e39028/cf1: appending metadata at 1675785243449 (+3 ms)Flushing 1e613dcf57a1f9c06070a32173e39028/cf1: closing flushed file at 1675785243449Flushing 1e613dcf57a1f9c06070a32173e39028/cf2: creating writer at 1675785243468 (+19 ms)Flushing 1e613dcf57a1f9c06070a32173e39028/cf2: appending metadata at 1675785243471 (+3 ms)Flushing 1e613dcf57a1f9c06070a32173e39028/cf2: closing flushed file at 1675785243471Flushing 1e613dcf57a1f9c06070a32173e39028/cf1: reopening flushed file at 1675785243488 (+17 ms)Flushing 1e613dcf57a1f9c06070a32173e39028/cf2: reopening flushed file at 1675785243501 (+13 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1e613dcf57a1f9c06070a32173e39028 in 64ms, sequenceid=82, compaction requested=true at 1675785243509 (+8 ms)Writing region close event to WAL at 1675785243523 (+14 ms)Running coprocessor post-close hooks at 1675785243527 (+4 ms)Closed at 1675785243528 (+1 ms) 2023-02-07 15:54:03,530 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,531 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=1e613dcf57a1f9c06070a32173e39028, regionState=CLOSED 2023-02-07 15:54:03,532 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675785243531"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785243531"}]},"ts":"1675785243531"} 2023-02-07 15:54:03,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-07 15:54:03,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 1e613dcf57a1f9c06070a32173e39028, server=jenkins-hbase12.apache.org,35657,1675785236854 in 249 msec 2023-02-07 15:54:03,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-07 15:54:03,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1e613dcf57a1f9c06070a32173e39028, UNASSIGN in 258 msec 2023-02-07 15:54:03,557 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=1e613dcf57a1f9c06070a32173e39028, threads=14 2023-02-07 15:54:03,558 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/31b897ea219d4f8c9f8f6298a47c32a2 for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,558 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/41a5bd1025fd4c93816f3f1d0b0e4e7c for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,558 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/5dd80339a63f49cdae13cfe694bd4144 for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,559 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/74d0eb3700ef440d9a093976ba929bfa for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,559 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/812659b3292944e6a9024715a8a9a148 for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,559 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/98775db67ca64b4e90613e8973fdbeb7 for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,559 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/fb60bbc8741c4c9181107980b9b3bc10 for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,559 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/039f958077d545baba1659650978773c for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,559 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/0cc2026840b04973a4109de67a8ebefd for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,560 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/278f7a6eeae14592a3a39b2bb040a948 for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,560 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/5ba564e3bd7e4a38bf3f64187c686c6a for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,560 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/6c8f680c3c3f45d9a8e1e35445f4bbbe for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,560 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/9e05977f40a2406d84575de48fd21a0b for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,561 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/d7a6d58e70ab457bbe8f6f47977d9c1e for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,583 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/41a5bd1025fd4c93816f3f1d0b0e4e7c, top=true 2023-02-07 15:54:03,595 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/039f958077d545baba1659650978773c, top=true 2023-02-07 15:54:03,595 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/31b897ea219d4f8c9f8f6298a47c32a2, top=true 2023-02-07 15:54:03,598 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/d7a6d58e70ab457bbe8f6f47977d9c1e, top=true 2023-02-07 15:54:03,614 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/6c8f680c3c3f45d9a8e1e35445f4bbbe, top=true 2023-02-07 15:54:03,618 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/98775db67ca64b4e90613e8973fdbeb7, top=true 2023-02-07 15:54:03,636 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.splits/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-d7a6d58e70ab457bbe8f6f47977d9c1e for child: 7cc858c64b5de4b5ff9aa3f61d757dee, parent: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,636 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/d7a6d58e70ab457bbe8f6f47977d9c1e for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,672 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.splits/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-41a5bd1025fd4c93816f3f1d0b0e4e7c for child: 7cc858c64b5de4b5ff9aa3f61d757dee, parent: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,681 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.splits/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-31b897ea219d4f8c9f8f6298a47c32a2 for child: 7cc858c64b5de4b5ff9aa3f61d757dee, parent: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,681 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/41a5bd1025fd4c93816f3f1d0b0e4e7c for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,691 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.splits/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-039f958077d545baba1659650978773c for child: 7cc858c64b5de4b5ff9aa3f61d757dee, parent: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,681 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/31b897ea219d4f8c9f8f6298a47c32a2 for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,693 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/039f958077d545baba1659650978773c for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,716 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.splits/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-6c8f680c3c3f45d9a8e1e35445f4bbbe for child: 7cc858c64b5de4b5ff9aa3f61d757dee, parent: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,717 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/6c8f680c3c3f45d9a8e1e35445f4bbbe for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,721 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/.splits/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-98775db67ca64b4e90613e8973fdbeb7 for child: 7cc858c64b5de4b5ff9aa3f61d757dee, parent: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,730 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/98775db67ca64b4e90613e8973fdbeb7 for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,801 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/9e05977f40a2406d84575de48fd21a0b for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,806 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/278f7a6eeae14592a3a39b2bb040a948 for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,806 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/fb60bbc8741c4c9181107980b9b3bc10 for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,813 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/74d0eb3700ef440d9a093976ba929bfa for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,835 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/812659b3292944e6a9024715a8a9a148 for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,836 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/0cc2026840b04973a4109de67a8ebefd for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:03,838 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/5dd80339a63f49cdae13cfe694bd4144 for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:04,142 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/5ba564e3bd7e4a38bf3f64187c686c6a for region: 1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:04,142 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 1e613dcf57a1f9c06070a32173e39028 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-07 15:54:04,203 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 15:54:04,206 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 15:54:04,209 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675785244209"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1675785244209"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1675785244209"}]},"ts":"1675785244209"} 2023-02-07 15:54:04,210 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675785244209"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785244209"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785244209"}]},"ts":"1675785244209"} 2023-02-07 15:54:04,210 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675785244209"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785244209"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785244209"}]},"ts":"1675785244209"} 2023-02-07 15:54:04,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=065cd7fcb400a33afc751c65a91aa196, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cc858c64b5de4b5ff9aa3f61d757dee, ASSIGN}] 2023-02-07 15:54:04,258 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cc858c64b5de4b5ff9aa3f61d757dee, ASSIGN 2023-02-07 15:54:04,259 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cc858c64b5de4b5ff9aa3f61d757dee, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,35657,1675785236854; forceNewPlan=false, retain=false 2023-02-07 15:54:04,261 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=065cd7fcb400a33afc751c65a91aa196, ASSIGN 2023-02-07 15:54:04,262 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=065cd7fcb400a33afc751c65a91aa196, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,35657,1675785236854; forceNewPlan=false, retain=false 2023-02-07 15:54:04,356 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 15:54:04,410 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 15:54:04,411 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=065cd7fcb400a33afc751c65a91aa196, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:04,411 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675785244411"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785244411"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785244411"}]},"ts":"1675785244411"} 2023-02-07 15:54:04,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 065cd7fcb400a33afc751c65a91aa196, server=jenkins-hbase12.apache.org,35657,1675785236854}] 2023-02-07 15:54:04,426 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=7cc858c64b5de4b5ff9aa3f61d757dee, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:04,427 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675785244426"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785244426"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785244426"}]},"ts":"1675785244426"} 2023-02-07 15:54:04,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 7cc858c64b5de4b5ff9aa3f61d757dee, server=jenkins-hbase12.apache.org,35657,1675785236854}] 2023-02-07 15:54:04,432 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-07 15:54:04,433 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-07 15:54:04,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee. 2023-02-07 15:54:04,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7cc858c64b5de4b5ff9aa3f61d757dee, NAME => 'testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.', STARTKEY => '26', ENDKEY => ''} 2023-02-07 15:54:04,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:04,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:04,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:04,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:04,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:04,581 INFO [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:04,582 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1 2023-02-07 15:54:04,582 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1 2023-02-07 15:54:04,583 INFO [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7cc858c64b5de4b5ff9aa3f61d757dee columnFamilyName cf1 2023-02-07 15:54:04,617 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/0cc2026840b04973a4109de67a8ebefd.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/0cc2026840b04973a4109de67a8ebefd-top 2023-02-07 15:54:04,627 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/278f7a6eeae14592a3a39b2bb040a948.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/278f7a6eeae14592a3a39b2bb040a948-top 2023-02-07 15:54:04,636 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/5ba564e3bd7e4a38bf3f64187c686c6a.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/5ba564e3bd7e4a38bf3f64187c686c6a-top 2023-02-07 15:54:04,645 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/9e05977f40a2406d84575de48fd21a0b.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/9e05977f40a2406d84575de48fd21a0b-top 2023-02-07 15:54:04,652 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-039f958077d545baba1659650978773c 2023-02-07 15:54:04,659 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-6c8f680c3c3f45d9a8e1e35445f4bbbe 2023-02-07 15:54:04,666 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-d7a6d58e70ab457bbe8f6f47977d9c1e 2023-02-07 15:54:04,666 INFO [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(310): Store=7cc858c64b5de4b5ff9aa3f61d757dee/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:04,667 INFO [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:04,668 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2 2023-02-07 15:54:04,668 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2 2023-02-07 15:54:04,669 INFO [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7cc858c64b5de4b5ff9aa3f61d757dee columnFamilyName cf2 2023-02-07 15:54:04,688 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/5dd80339a63f49cdae13cfe694bd4144.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/5dd80339a63f49cdae13cfe694bd4144-top 2023-02-07 15:54:04,695 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/74d0eb3700ef440d9a093976ba929bfa.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/74d0eb3700ef440d9a093976ba929bfa-top 2023-02-07 15:54:04,702 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/812659b3292944e6a9024715a8a9a148.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/812659b3292944e6a9024715a8a9a148-top 2023-02-07 15:54:04,710 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/fb60bbc8741c4c9181107980b9b3bc10.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/fb60bbc8741c4c9181107980b9b3bc10-top 2023-02-07 15:54:04,715 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-31b897ea219d4f8c9f8f6298a47c32a2 2023-02-07 15:54:04,721 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-41a5bd1025fd4c93816f3f1d0b0e4e7c 2023-02-07 15:54:04,728 DEBUG [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-98775db67ca64b4e90613e8973fdbeb7 2023-02-07 15:54:04,728 INFO [StoreOpener-7cc858c64b5de4b5ff9aa3f61d757dee-1] regionserver.HStore(310): Store=7cc858c64b5de4b5ff9aa3f61d757dee/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:04,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:04,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:04,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:04,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:04,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 7cc858c64b5de4b5ff9aa3f61d757dee; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66723871, jitterRate=-0.005736842751502991}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:04,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 7cc858c64b5de4b5ff9aa3f61d757dee: Running coprocessor pre-open hook at 1675785244580Writing region info on filesystem at 1675785244580Initializing all the Stores at 1675785244581 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785244581Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785244581Cleaning up temporary data from old regions at 1675785244732 (+151 ms)Cleaning up detritus from prior splits at 1675785244733 (+1 ms)Running coprocessor post-open hooks at 1675785244737 (+4 ms)Region opened successfully at 1675785244737 2023-02-07 15:54:04,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee., pid=18, masterSystemTime=1675785244573 2023-02-07 15:54:04,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:04,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:54:04,742 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:54:04,742 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:54:04,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee. 2023-02-07 15:54:04,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee. 2023-02-07 15:54:04,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196. 2023-02-07 15:54:04,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 065cd7fcb400a33afc751c65a91aa196, NAME => 'testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.', STARTKEY => '', ENDKEY => '26'} 2023-02-07 15:54:04,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:04,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 065cd7fcb400a33afc751c65a91aa196 2023-02-07 15:54:04,746 INFO [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee. 2023-02-07 15:54:04,746 INFO [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee. 2023-02-07 15:54:04,746 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HStore(1912): 7cc858c64b5de4b5ff9aa3f61d757dee/cf2 is initiating minor compaction (all files) 2023-02-07 15:54:04,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:04,746 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=7cc858c64b5de4b5ff9aa3f61d757dee, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:04,746 INFO [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 7cc858c64b5de4b5ff9aa3f61d757dee/cf2 in testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee. 2023-02-07 15:54:04,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 065cd7fcb400a33afc751c65a91aa196 2023-02-07 15:54:04,746 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HStore(1912): 7cc858c64b5de4b5ff9aa3f61d757dee/cf1 is initiating minor compaction (all files) 2023-02-07 15:54:04,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 065cd7fcb400a33afc751c65a91aa196 2023-02-07 15:54:04,746 INFO [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7cc858c64b5de4b5ff9aa3f61d757dee/cf1 in testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee. 2023-02-07 15:54:04,746 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675785244746"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785244746"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785244746"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785244746"}]},"ts":"1675785244746"} 2023-02-07 15:54:04,747 INFO [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/74d0eb3700ef440d9a093976ba929bfa.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/74d0eb3700ef440d9a093976ba929bfa-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/fb60bbc8741c4c9181107980b9b3bc10.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/fb60bbc8741c4c9181107980b9b3bc10-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/5dd80339a63f49cdae13cfe694bd4144.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/5dd80339a63f49cdae13cfe694bd4144-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/812659b3292944e6a9024715a8a9a148.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/812659b3292944e6a9024715a8a9a148-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-98775db67ca64b4e90613e8973fdbeb7, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-41a5bd1025fd4c93816f3f1d0b0e4e7c, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-31b897ea219d4f8c9f8f6298a47c32a2] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/.tmp, totalSize=34.9 K 2023-02-07 15:54:04,747 INFO [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/5ba564e3bd7e4a38bf3f64187c686c6a.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/5ba564e3bd7e4a38bf3f64187c686c6a-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/9e05977f40a2406d84575de48fd21a0b.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/9e05977f40a2406d84575de48fd21a0b-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/0cc2026840b04973a4109de67a8ebefd.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/0cc2026840b04973a4109de67a8ebefd-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/278f7a6eeae14592a3a39b2bb040a948.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/278f7a6eeae14592a3a39b2bb040a948-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-6c8f680c3c3f45d9a8e1e35445f4bbbe, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-d7a6d58e70ab457bbe8f6f47977d9c1e, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-039f958077d545baba1659650978773c] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/.tmp, totalSize=34.9 K 2023-02-07 15:54:04,749 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.Compactor(207): Compacting 74d0eb3700ef440d9a093976ba929bfa.1e613dcf57a1f9c06070a32173e39028, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675785240854 2023-02-07 15:54:04,749 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.Compactor(207): Compacting 5ba564e3bd7e4a38bf3f64187c686c6a.1e613dcf57a1f9c06070a32173e39028, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675785240854 2023-02-07 15:54:04,751 INFO [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 065cd7fcb400a33afc751c65a91aa196 2023-02-07 15:54:04,752 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.Compactor(207): Compacting fb60bbc8741c4c9181107980b9b3bc10.1e613dcf57a1f9c06070a32173e39028, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675785241383 2023-02-07 15:54:04,753 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.Compactor(207): Compacting 5dd80339a63f49cdae13cfe694bd4144.1e613dcf57a1f9c06070a32173e39028, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675785241719 2023-02-07 15:54:04,754 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.Compactor(207): Compacting 9e05977f40a2406d84575de48fd21a0b.1e613dcf57a1f9c06070a32173e39028, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675785241383 2023-02-07 15:54:04,754 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.Compactor(207): Compacting 812659b3292944e6a9024715a8a9a148.1e613dcf57a1f9c06070a32173e39028, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675785242025 2023-02-07 15:54:04,755 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-98775db67ca64b4e90613e8973fdbeb7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785242397 2023-02-07 15:54:04,755 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.Compactor(207): Compacting 0cc2026840b04973a4109de67a8ebefd.1e613dcf57a1f9c06070a32173e39028, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675785241719 2023-02-07 15:54:04,756 DEBUG [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1 2023-02-07 15:54:04,756 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-41a5bd1025fd4c93816f3f1d0b0e4e7c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785242754 2023-02-07 15:54:04,756 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.Compactor(207): Compacting 278f7a6eeae14592a3a39b2bb040a948.1e613dcf57a1f9c06070a32173e39028, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675785242025 2023-02-07 15:54:04,756 DEBUG [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1 2023-02-07 15:54:04,756 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-6c8f680c3c3f45d9a8e1e35445f4bbbe, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785242397 2023-02-07 15:54:04,756 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-31b897ea219d4f8c9f8f6298a47c32a2, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675785243240 2023-02-07 15:54:04,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-02-07 15:54:04,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 7cc858c64b5de4b5ff9aa3f61d757dee, server=jenkins-hbase12.apache.org,35657,1675785236854 in 322 msec 2023-02-07 15:54:04,757 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-d7a6d58e70ab457bbe8f6f47977d9c1e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785242754 2023-02-07 15:54:04,759 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-039f958077d545baba1659650978773c, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675785243240 2023-02-07 15:54:04,760 INFO [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 065cd7fcb400a33afc751c65a91aa196 columnFamilyName cf1 2023-02-07 15:54:04,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cc858c64b5de4b5ff9aa3f61d757dee, ASSIGN in 501 msec 2023-02-07 15:54:04,796 DEBUG [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/0cc2026840b04973a4109de67a8ebefd.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/0cc2026840b04973a4109de67a8ebefd-bottom 2023-02-07 15:54:04,803 INFO [RS:1;jenkins-hbase12:35657-longCompactions-0] throttle.PressureAwareThroughputController(145): 7cc858c64b5de4b5ff9aa3f61d757dee#cf2#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:04,803 INFO [RS:1;jenkins-hbase12:35657-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7cc858c64b5de4b5ff9aa3f61d757dee#cf1#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:04,807 DEBUG [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/278f7a6eeae14592a3a39b2bb040a948.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/278f7a6eeae14592a3a39b2bb040a948-bottom 2023-02-07 15:54:04,813 DEBUG [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/5ba564e3bd7e4a38bf3f64187c686c6a.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/5ba564e3bd7e4a38bf3f64187c686c6a-bottom 2023-02-07 15:54:04,819 DEBUG [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/9e05977f40a2406d84575de48fd21a0b.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/9e05977f40a2406d84575de48fd21a0b-bottom 2023-02-07 15:54:04,820 INFO [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] regionserver.HStore(310): Store=065cd7fcb400a33afc751c65a91aa196/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:04,820 INFO [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 065cd7fcb400a33afc751c65a91aa196 2023-02-07 15:54:04,821 DEBUG [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2 2023-02-07 15:54:04,822 DEBUG [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2 2023-02-07 15:54:04,822 INFO [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 065cd7fcb400a33afc751c65a91aa196 columnFamilyName cf2 2023-02-07 15:54:04,829 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/.tmp/cf2/e4caef88b87745f98460fc05e1389025 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/e4caef88b87745f98460fc05e1389025 2023-02-07 15:54:04,839 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/.tmp/cf1/6bd709c2e26043018a852d0b493c1a7e as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/6bd709c2e26043018a852d0b493c1a7e 2023-02-07 15:54:04,845 DEBUG [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/5dd80339a63f49cdae13cfe694bd4144.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/5dd80339a63f49cdae13cfe694bd4144-bottom 2023-02-07 15:54:04,852 DEBUG [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/74d0eb3700ef440d9a093976ba929bfa.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/74d0eb3700ef440d9a093976ba929bfa-bottom 2023-02-07 15:54:04,855 INFO [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7cc858c64b5de4b5ff9aa3f61d757dee/cf2 of 7cc858c64b5de4b5ff9aa3f61d757dee into e4caef88b87745f98460fc05e1389025(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:04,855 INFO [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7cc858c64b5de4b5ff9aa3f61d757dee/cf1 of 7cc858c64b5de4b5ff9aa3f61d757dee into 6bd709c2e26043018a852d0b493c1a7e(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:04,855 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7cc858c64b5de4b5ff9aa3f61d757dee: 2023-02-07 15:54:04,855 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7cc858c64b5de4b5ff9aa3f61d757dee: 2023-02-07 15:54:04,855 INFO [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee., storeName=7cc858c64b5de4b5ff9aa3f61d757dee/cf2, priority=9, startTime=1675785244742; duration=0sec 2023-02-07 15:54:04,855 INFO [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee., storeName=7cc858c64b5de4b5ff9aa3f61d757dee/cf1, priority=9, startTime=1675785244741; duration=0sec 2023-02-07 15:54:04,855 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:04,855 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:04,858 DEBUG [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/812659b3292944e6a9024715a8a9a148.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/812659b3292944e6a9024715a8a9a148-bottom 2023-02-07 15:54:04,863 DEBUG [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/fb60bbc8741c4c9181107980b9b3bc10.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/fb60bbc8741c4c9181107980b9b3bc10-bottom 2023-02-07 15:54:04,863 INFO [StoreOpener-065cd7fcb400a33afc751c65a91aa196-1] regionserver.HStore(310): Store=065cd7fcb400a33afc751c65a91aa196/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:04,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196 2023-02-07 15:54:04,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196 2023-02-07 15:54:04,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:04,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 065cd7fcb400a33afc751c65a91aa196 2023-02-07 15:54:04,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 065cd7fcb400a33afc751c65a91aa196; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71119088, jitterRate=0.05975699424743652}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:04,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 065cd7fcb400a33afc751c65a91aa196: Running coprocessor pre-open hook at 1675785244746Writing region info on filesystem at 1675785244746Initializing all the Stores at 1675785244750 (+4 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785244750Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785244750Cleaning up temporary data from old regions at 1675785244867 (+117 ms)Cleaning up detritus from prior splits at 1675785244867Running coprocessor post-open hooks at 1675785244871 (+4 ms)Region opened successfully at 1675785244871 2023-02-07 15:54:04,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196., pid=17, masterSystemTime=1675785244573 2023-02-07 15:54:04,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:04,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:54:04,873 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-07 15:54:04,873 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-07 15:54:04,873 INFO [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196. 2023-02-07 15:54:04,873 INFO [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196. 2023-02-07 15:54:04,873 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HStore(1912): 065cd7fcb400a33afc751c65a91aa196/cf1 is initiating minor compaction (all files) 2023-02-07 15:54:04,873 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HStore(1912): 065cd7fcb400a33afc751c65a91aa196/cf2 is initiating minor compaction (all files) 2023-02-07 15:54:04,873 INFO [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 065cd7fcb400a33afc751c65a91aa196/cf1 in testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196. 2023-02-07 15:54:04,873 INFO [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 065cd7fcb400a33afc751c65a91aa196/cf2 in testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196. 2023-02-07 15:54:04,874 INFO [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/74d0eb3700ef440d9a093976ba929bfa.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/74d0eb3700ef440d9a093976ba929bfa-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/fb60bbc8741c4c9181107980b9b3bc10.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/fb60bbc8741c4c9181107980b9b3bc10-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/5dd80339a63f49cdae13cfe694bd4144.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/5dd80339a63f49cdae13cfe694bd4144-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/812659b3292944e6a9024715a8a9a148.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/812659b3292944e6a9024715a8a9a148-bottom] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/.tmp, totalSize=19.9 K 2023-02-07 15:54:04,874 INFO [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/5ba564e3bd7e4a38bf3f64187c686c6a.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/5ba564e3bd7e4a38bf3f64187c686c6a-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/9e05977f40a2406d84575de48fd21a0b.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/9e05977f40a2406d84575de48fd21a0b-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/0cc2026840b04973a4109de67a8ebefd.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/0cc2026840b04973a4109de67a8ebefd-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/278f7a6eeae14592a3a39b2bb040a948.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/278f7a6eeae14592a3a39b2bb040a948-bottom] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/.tmp, totalSize=19.9 K 2023-02-07 15:54:04,875 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.Compactor(207): Compacting 74d0eb3700ef440d9a093976ba929bfa.1e613dcf57a1f9c06070a32173e39028, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785240854 2023-02-07 15:54:04,875 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.Compactor(207): Compacting 5ba564e3bd7e4a38bf3f64187c686c6a.1e613dcf57a1f9c06070a32173e39028, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785240854 2023-02-07 15:54:04,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196. 2023-02-07 15:54:04,876 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.Compactor(207): Compacting fb60bbc8741c4c9181107980b9b3bc10.1e613dcf57a1f9c06070a32173e39028, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785241383 2023-02-07 15:54:04,876 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.Compactor(207): Compacting 9e05977f40a2406d84575de48fd21a0b.1e613dcf57a1f9c06070a32173e39028, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785241383 2023-02-07 15:54:04,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196. 2023-02-07 15:54:04,876 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.Compactor(207): Compacting 5dd80339a63f49cdae13cfe694bd4144.1e613dcf57a1f9c06070a32173e39028, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785241719 2023-02-07 15:54:04,877 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=065cd7fcb400a33afc751c65a91aa196, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:04,877 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.Compactor(207): Compacting 0cc2026840b04973a4109de67a8ebefd.1e613dcf57a1f9c06070a32173e39028, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785241719 2023-02-07 15:54:04,877 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675785244877"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785244877"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785244877"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785244877"}]},"ts":"1675785244877"} 2023-02-07 15:54:04,877 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] compactions.Compactor(207): Compacting 812659b3292944e6a9024715a8a9a148.1e613dcf57a1f9c06070a32173e39028, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785242025 2023-02-07 15:54:04,877 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] compactions.Compactor(207): Compacting 278f7a6eeae14592a3a39b2bb040a948.1e613dcf57a1f9c06070a32173e39028, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785242025 2023-02-07 15:54:04,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-07 15:54:04,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 065cd7fcb400a33afc751c65a91aa196, server=jenkins-hbase12.apache.org,35657,1675785236854 in 467 msec 2023-02-07 15:54:04,889 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-02-07 15:54:04,889 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=065cd7fcb400a33afc751c65a91aa196, ASSIGN in 629 msec 2023-02-07 15:54:04,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee in 1.6260 sec 2023-02-07 15:54:04,892 INFO [RS:1;jenkins-hbase12:35657-longCompactions-0] throttle.PressureAwareThroughputController(145): 065cd7fcb400a33afc751c65a91aa196#cf2#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:04,892 INFO [RS:1;jenkins-hbase12:35657-shortCompactions-0] throttle.PressureAwareThroughputController(145): 065cd7fcb400a33afc751c65a91aa196#cf1#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:04,911 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/.tmp/cf1/3656e9e581b14bb596d6bc8203892ccf as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/3656e9e581b14bb596d6bc8203892ccf 2023-02-07 15:54:04,911 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/.tmp/cf2/f7763577e59948119ef00e7a53f64eb1 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/f7763577e59948119ef00e7a53f64eb1 2023-02-07 15:54:04,918 INFO [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 065cd7fcb400a33afc751c65a91aa196/cf2 of 065cd7fcb400a33afc751c65a91aa196 into f7763577e59948119ef00e7a53f64eb1(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:04,918 INFO [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 065cd7fcb400a33afc751c65a91aa196/cf1 of 065cd7fcb400a33afc751c65a91aa196 into 3656e9e581b14bb596d6bc8203892ccf(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:04,918 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 065cd7fcb400a33afc751c65a91aa196: 2023-02-07 15:54:04,918 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 065cd7fcb400a33afc751c65a91aa196: 2023-02-07 15:54:04,918 INFO [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196., storeName=065cd7fcb400a33afc751c65a91aa196/cf2, priority=12, startTime=1675785244873; duration=0sec 2023-02-07 15:54:04,918 INFO [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196., storeName=065cd7fcb400a33afc751c65a91aa196/cf1, priority=12, startTime=1675785244872; duration=0sec 2023-02-07 15:54:04,918 DEBUG [RS:1;jenkins-hbase12:35657-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:04,918 DEBUG [RS:1;jenkins-hbase12:35657-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:05,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35657] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 136.243.104.168:60662 deadline: 1675785305023, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028. is not online on jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:05,156 DEBUG [Listener at localhost.localdomain/41965] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-07 15:54:05,169 INFO [Listener at localhost.localdomain/41965] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-07 15:54:05,171 DEBUG [Listener at localhost.localdomain/41965] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-07 15:54:05,201 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:05,202 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:05,202 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:05,210 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:05,210 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:05,211 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-07 15:54:05,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegionUnevenDaughter 2023-02-07 15:54:05,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:05,233 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785245233"}]},"ts":"1675785245233"} 2023-02-07 15:54:05,236 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-07 15:54:05,238 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:05,238 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:05,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-07 15:54:05,297 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-07 15:54:05,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=065cd7fcb400a33afc751c65a91aa196, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cc858c64b5de4b5ff9aa3f61d757dee, UNASSIGN}] 2023-02-07 15:54:05,301 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cc858c64b5de4b5ff9aa3f61d757dee, UNASSIGN 2023-02-07 15:54:05,302 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7cc858c64b5de4b5ff9aa3f61d757dee, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:05,302 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675785245302"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785245302"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785245302"}]},"ts":"1675785245302"} 2023-02-07 15:54:05,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 7cc858c64b5de4b5ff9aa3f61d757dee, server=jenkins-hbase12.apache.org,35657,1675785236854}] 2023-02-07 15:54:05,308 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=065cd7fcb400a33afc751c65a91aa196, UNASSIGN 2023-02-07 15:54:05,309 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=065cd7fcb400a33afc751c65a91aa196, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:05,309 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675785245309"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785245309"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785245309"}]},"ts":"1675785245309"} 2023-02-07 15:54:05,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 065cd7fcb400a33afc751c65a91aa196, server=jenkins-hbase12.apache.org,35657,1675785236854}] 2023-02-07 15:54:05,344 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:05,345 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:05,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-07 15:54:05,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:05,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 7cc858c64b5de4b5ff9aa3f61d757dee, disabling compactions & flushes 2023-02-07 15:54:05,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee. 2023-02-07 15:54:05,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee. 2023-02-07 15:54:05,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee. after waiting 0 ms 2023-02-07 15:54:05,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee. 2023-02-07 15:54:05,468 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/5ba564e3bd7e4a38bf3f64187c686c6a.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/5ba564e3bd7e4a38bf3f64187c686c6a-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/9e05977f40a2406d84575de48fd21a0b.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/9e05977f40a2406d84575de48fd21a0b-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/0cc2026840b04973a4109de67a8ebefd.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/0cc2026840b04973a4109de67a8ebefd-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/278f7a6eeae14592a3a39b2bb040a948.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/278f7a6eeae14592a3a39b2bb040a948-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-6c8f680c3c3f45d9a8e1e35445f4bbbe, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-d7a6d58e70ab457bbe8f6f47977d9c1e, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-039f958077d545baba1659650978773c] to archive 2023-02-07 15:54:05,469 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:05,476 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/5ba564e3bd7e4a38bf3f64187c686c6a.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/5ba564e3bd7e4a38bf3f64187c686c6a.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,478 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/9e05977f40a2406d84575de48fd21a0b.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/9e05977f40a2406d84575de48fd21a0b.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,480 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/0cc2026840b04973a4109de67a8ebefd.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/0cc2026840b04973a4109de67a8ebefd.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,481 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/278f7a6eeae14592a3a39b2bb040a948.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/278f7a6eeae14592a3a39b2bb040a948.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,483 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-6c8f680c3c3f45d9a8e1e35445f4bbbe to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-6c8f680c3c3f45d9a8e1e35445f4bbbe 2023-02-07 15:54:05,485 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-d7a6d58e70ab457bbe8f6f47977d9c1e to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-d7a6d58e70ab457bbe8f6f47977d9c1e 2023-02-07 15:54:05,487 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-039f958077d545baba1659650978773c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-039f958077d545baba1659650978773c 2023-02-07 15:54:05,508 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/74d0eb3700ef440d9a093976ba929bfa.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/74d0eb3700ef440d9a093976ba929bfa-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/fb60bbc8741c4c9181107980b9b3bc10.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/fb60bbc8741c4c9181107980b9b3bc10-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/5dd80339a63f49cdae13cfe694bd4144.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/5dd80339a63f49cdae13cfe694bd4144-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/812659b3292944e6a9024715a8a9a148.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/812659b3292944e6a9024715a8a9a148-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-98775db67ca64b4e90613e8973fdbeb7, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-41a5bd1025fd4c93816f3f1d0b0e4e7c, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-31b897ea219d4f8c9f8f6298a47c32a2] to archive 2023-02-07 15:54:05,509 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:05,511 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/74d0eb3700ef440d9a093976ba929bfa.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/74d0eb3700ef440d9a093976ba929bfa.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,513 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/fb60bbc8741c4c9181107980b9b3bc10.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/fb60bbc8741c4c9181107980b9b3bc10.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,515 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/5dd80339a63f49cdae13cfe694bd4144.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/5dd80339a63f49cdae13cfe694bd4144.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,517 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/812659b3292944e6a9024715a8a9a148.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/812659b3292944e6a9024715a8a9a148.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,519 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-98775db67ca64b4e90613e8973fdbeb7 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-98775db67ca64b4e90613e8973fdbeb7 2023-02-07 15:54:05,520 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-41a5bd1025fd4c93816f3f1d0b0e4e7c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-41a5bd1025fd4c93816f3f1d0b0e4e7c 2023-02-07 15:54:05,522 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-31b897ea219d4f8c9f8f6298a47c32a2 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/testSplitTableRegionUnevenDaughter=1e613dcf57a1f9c06070a32173e39028-31b897ea219d4f8c9f8f6298a47c32a2 2023-02-07 15:54:05,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 15:54:05,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:05,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee. 2023-02-07 15:54:05,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 7cc858c64b5de4b5ff9aa3f61d757dee: Waiting for close lock at 1675785245457Running coprocessor pre-close hooks at 1675785245457Disabling compacts and flushes for region at 1675785245457Disabling writes for close at 1675785245458 (+1 ms)Writing region close event to WAL at 1675785245523 (+65 ms)Running coprocessor post-close hooks at 1675785245527 (+4 ms)Closed at 1675785245528 (+1 ms) 2023-02-07 15:54:05,530 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:05,530 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 065cd7fcb400a33afc751c65a91aa196 2023-02-07 15:54:05,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 065cd7fcb400a33afc751c65a91aa196, disabling compactions & flushes 2023-02-07 15:54:05,531 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196. 2023-02-07 15:54:05,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196. 2023-02-07 15:54:05,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196. after waiting 0 ms 2023-02-07 15:54:05,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196. 2023-02-07 15:54:05,532 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7cc858c64b5de4b5ff9aa3f61d757dee, regionState=CLOSED 2023-02-07 15:54:05,532 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675785245531"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785245531"}]},"ts":"1675785245531"} 2023-02-07 15:54:05,532 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/5ba564e3bd7e4a38bf3f64187c686c6a.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/5ba564e3bd7e4a38bf3f64187c686c6a-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/9e05977f40a2406d84575de48fd21a0b.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/9e05977f40a2406d84575de48fd21a0b-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/0cc2026840b04973a4109de67a8ebefd.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/0cc2026840b04973a4109de67a8ebefd-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/278f7a6eeae14592a3a39b2bb040a948.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/278f7a6eeae14592a3a39b2bb040a948-bottom] to archive 2023-02-07 15:54:05,533 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:05,536 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/5ba564e3bd7e4a38bf3f64187c686c6a.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/5ba564e3bd7e4a38bf3f64187c686c6a.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-07 15:54:05,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 7cc858c64b5de4b5ff9aa3f61d757dee, server=jenkins-hbase12.apache.org,35657,1675785236854 in 231 msec 2023-02-07 15:54:05,539 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/9e05977f40a2406d84575de48fd21a0b.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/9e05977f40a2406d84575de48fd21a0b.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,540 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/0cc2026840b04973a4109de67a8ebefd.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/0cc2026840b04973a4109de67a8ebefd.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cc858c64b5de4b5ff9aa3f61d757dee, UNASSIGN in 240 msec 2023-02-07 15:54:05,542 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/278f7a6eeae14592a3a39b2bb040a948.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/278f7a6eeae14592a3a39b2bb040a948.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,544 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/74d0eb3700ef440d9a093976ba929bfa.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/74d0eb3700ef440d9a093976ba929bfa-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/fb60bbc8741c4c9181107980b9b3bc10.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/fb60bbc8741c4c9181107980b9b3bc10-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/5dd80339a63f49cdae13cfe694bd4144.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/5dd80339a63f49cdae13cfe694bd4144-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/812659b3292944e6a9024715a8a9a148.1e613dcf57a1f9c06070a32173e39028->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/812659b3292944e6a9024715a8a9a148-bottom] to archive 2023-02-07 15:54:05,545 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:05,546 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:05,546 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:05,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-07 15:54:05,548 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/74d0eb3700ef440d9a093976ba929bfa.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/74d0eb3700ef440d9a093976ba929bfa.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,550 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/fb60bbc8741c4c9181107980b9b3bc10.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/fb60bbc8741c4c9181107980b9b3bc10.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,551 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/5dd80339a63f49cdae13cfe694bd4144.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/5dd80339a63f49cdae13cfe694bd4144.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,552 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/812659b3292944e6a9024715a8a9a148.1e613dcf57a1f9c06070a32173e39028 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/812659b3292944e6a9024715a8a9a148.1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,557 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 15:54:05,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:05,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196. 2023-02-07 15:54:05,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 065cd7fcb400a33afc751c65a91aa196: Waiting for close lock at 1675785245531Running coprocessor pre-close hooks at 1675785245531Disabling compacts and flushes for region at 1675785245531Disabling writes for close at 1675785245531Writing region close event to WAL at 1675785245554 (+23 ms)Running coprocessor post-close hooks at 1675785245558 (+4 ms)Closed at 1675785245558 2023-02-07 15:54:05,561 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 065cd7fcb400a33afc751c65a91aa196 2023-02-07 15:54:05,562 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=065cd7fcb400a33afc751c65a91aa196, regionState=CLOSED 2023-02-07 15:54:05,562 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675785245562"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785245562"}]},"ts":"1675785245562"} 2023-02-07 15:54:05,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-07 15:54:05,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 065cd7fcb400a33afc751c65a91aa196, server=jenkins-hbase12.apache.org,35657,1675785236854 in 253 msec 2023-02-07 15:54:05,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-07 15:54:05,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=065cd7fcb400a33afc751c65a91aa196, UNASSIGN in 268 msec 2023-02-07 15:54:05,572 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785245572"}]},"ts":"1675785245572"} 2023-02-07 15:54:05,574 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-07 15:54:05,587 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-07 15:54:05,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 369 msec 2023-02-07 15:54:05,744 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-07 15:54:05,745 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-07 15:54:05,745 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:05,745 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-07 15:54:05,848 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:05,848 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:05,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-07 15:54:05,850 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-07 15:54:05,851 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:05,851 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:05,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegionUnevenDaughter 2023-02-07 15:54:05,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:05,863 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:05,865 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:05,865 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:05,866 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:05,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-07 15:54:05,873 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,873 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:05,873 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196 2023-02-07 15:54:05,876 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/recovered.edits] 2023-02-07 15:54:05,876 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/recovered.edits] 2023-02-07 15:54:05,876 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/recovered.edits] 2023-02-07 15:54:05,879 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/6bd709c2e26043018a852d0b493c1a7e to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf1/6bd709c2e26043018a852d0b493c1a7e 2023-02-07 15:54:05,880 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/3656e9e581b14bb596d6bc8203892ccf to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf1/3656e9e581b14bb596d6bc8203892ccf 2023-02-07 15:54:05,880 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/039f958077d545baba1659650978773c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/039f958077d545baba1659650978773c 2023-02-07 15:54:05,881 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/0cc2026840b04973a4109de67a8ebefd to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/0cc2026840b04973a4109de67a8ebefd 2023-02-07 15:54:05,882 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/e4caef88b87745f98460fc05e1389025 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/cf2/e4caef88b87745f98460fc05e1389025 2023-02-07 15:54:05,883 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/f7763577e59948119ef00e7a53f64eb1 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/cf2/f7763577e59948119ef00e7a53f64eb1 2023-02-07 15:54:05,883 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/278f7a6eeae14592a3a39b2bb040a948 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/278f7a6eeae14592a3a39b2bb040a948 2023-02-07 15:54:05,884 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/5ba564e3bd7e4a38bf3f64187c686c6a to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/5ba564e3bd7e4a38bf3f64187c686c6a 2023-02-07 15:54:05,886 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/recovered.edits/90.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196/recovered.edits/90.seqid 2023-02-07 15:54:05,887 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/6c8f680c3c3f45d9a8e1e35445f4bbbe to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/6c8f680c3c3f45d9a8e1e35445f4bbbe 2023-02-07 15:54:05,887 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/recovered.edits/90.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee/recovered.edits/90.seqid 2023-02-07 15:54:05,888 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/065cd7fcb400a33afc751c65a91aa196 2023-02-07 15:54:05,888 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:05,889 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/9e05977f40a2406d84575de48fd21a0b to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/9e05977f40a2406d84575de48fd21a0b 2023-02-07 15:54:05,890 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/d7a6d58e70ab457bbe8f6f47977d9c1e to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf1/d7a6d58e70ab457bbe8f6f47977d9c1e 2023-02-07 15:54:05,893 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/31b897ea219d4f8c9f8f6298a47c32a2 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/31b897ea219d4f8c9f8f6298a47c32a2 2023-02-07 15:54:05,895 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/41a5bd1025fd4c93816f3f1d0b0e4e7c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/41a5bd1025fd4c93816f3f1d0b0e4e7c 2023-02-07 15:54:05,897 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/5dd80339a63f49cdae13cfe694bd4144 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/5dd80339a63f49cdae13cfe694bd4144 2023-02-07 15:54:05,899 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/74d0eb3700ef440d9a093976ba929bfa to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/74d0eb3700ef440d9a093976ba929bfa 2023-02-07 15:54:05,900 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/812659b3292944e6a9024715a8a9a148 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/812659b3292944e6a9024715a8a9a148 2023-02-07 15:54:05,902 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/98775db67ca64b4e90613e8973fdbeb7 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/98775db67ca64b4e90613e8973fdbeb7 2023-02-07 15:54:05,903 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/fb60bbc8741c4c9181107980b9b3bc10 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/cf2/fb60bbc8741c4c9181107980b9b3bc10 2023-02-07 15:54:05,906 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/recovered.edits/85.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028/recovered.edits/85.seqid 2023-02-07 15:54:05,907 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionUnevenDaughter/1e613dcf57a1f9c06070a32173e39028 2023-02-07 15:54:05,907 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-07 15:54:05,910 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:05,935 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-07 15:54:05,967 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:05,967 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:05,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-07 15:54:05,972 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-07 15:54:05,974 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:05,974 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-07 15:54:05,974 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785245974"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:05,974 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785245974"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:05,974 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785245974"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:05,977 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 15:54:05,977 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1e613dcf57a1f9c06070a32173e39028, NAME => 'testSplitTableRegionUnevenDaughter,,1675785240091.1e613dcf57a1f9c06070a32173e39028.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 065cd7fcb400a33afc751c65a91aa196, NAME => 'testSplitTableRegionUnevenDaughter,,1675785243262.065cd7fcb400a33afc751c65a91aa196.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 7cc858c64b5de4b5ff9aa3f61d757dee, NAME => 'testSplitTableRegionUnevenDaughter,26,1675785243262.7cc858c64b5de4b5ff9aa3f61d757dee.', STARTKEY => '26', ENDKEY => ''}] 2023-02-07 15:54:05,977 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-07 15:54:05,978 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675785245978"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:05,980 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-07 15:54:06,030 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:06,032 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 174 msec 2023-02-07 15:54:06,168 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:06,169 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:06,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-07 15:54:06,171 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-07 15:54:06,187 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=395 (was 362) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa17a051-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:33924 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:33824 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:33624 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:33816 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:33826 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:51598 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa17a051-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:51624 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa17a051-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa17a051-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:51592 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:33550 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:51582 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:51190 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:51612 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa17a051-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:33994 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:38424 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:34254 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1887562514_15 at /127.0.0.1:33784 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase12:35657-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=849 (was 700) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=394 (was 383) - SystemLoadAverage LEAK? -, ProcessCount=173 (was 173), AvailableMemoryMB=3317 (was 3373) 2023-02-07 15:54:06,201 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=395, OpenFileDescriptor=849, MaxFileDescriptor=60000, SystemLoadAverage=394, ProcessCount=173, AvailableMemoryMB=3316 2023-02-07 15:54:06,202 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:06,202 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:06,202 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:06,241 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 15:54:06,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 15:54:06,243 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:06,245 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 15:54:06,247 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 15:54:06,249 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:06,249 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543 empty. 2023-02-07 15:54:06,250 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:06,250 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-07 15:54:06,262 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-07 15:54:06,263 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => d7d21b347a5059d92fcfbcd4fc307543, NAME => 'testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp 2023-02-07 15:54:06,271 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:06,271 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing d7d21b347a5059d92fcfbcd4fc307543, disabling compactions & flushes 2023-02-07 15:54:06,272 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:06,272 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:06,272 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. after waiting 0 ms 2023-02-07 15:54:06,272 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:06,272 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:06,272 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for d7d21b347a5059d92fcfbcd4fc307543: Waiting for close lock at 1675785246271Disabling compacts and flushes for region at 1675785246271Disabling writes for close at 1675785246272 (+1 ms)Writing region close event to WAL at 1675785246272Closed at 1675785246272 2023-02-07 15:54:06,274 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 15:54:06,275 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785246275"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785246275"}]},"ts":"1675785246275"} 2023-02-07 15:54:06,277 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 15:54:06,278 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 15:54:06,278 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785246278"}]},"ts":"1675785246278"} 2023-02-07 15:54:06,280 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-07 15:54:06,307 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:54:06,309 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:54:06,309 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:54:06,309 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:54:06,309 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 15:54:06,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, ASSIGN}] 2023-02-07 15:54:06,312 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, ASSIGN 2023-02-07 15:54:06,313 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:54:06,464 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:54:06,465 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=d7d21b347a5059d92fcfbcd4fc307543, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,465 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785246465"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785246465"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785246465"}]},"ts":"1675785246465"} 2023-02-07 15:54:06,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:06,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:06,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d7d21b347a5059d92fcfbcd4fc307543, NAME => 'testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:54:06,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:06,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:06,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:06,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:06,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:06,625 INFO [StoreOpener-d7d21b347a5059d92fcfbcd4fc307543-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:06,627 DEBUG [StoreOpener-d7d21b347a5059d92fcfbcd4fc307543-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1 2023-02-07 15:54:06,627 DEBUG [StoreOpener-d7d21b347a5059d92fcfbcd4fc307543-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1 2023-02-07 15:54:06,627 INFO [StoreOpener-d7d21b347a5059d92fcfbcd4fc307543-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d7d21b347a5059d92fcfbcd4fc307543 columnFamilyName cf1 2023-02-07 15:54:06,628 INFO [StoreOpener-d7d21b347a5059d92fcfbcd4fc307543-1] regionserver.HStore(310): Store=d7d21b347a5059d92fcfbcd4fc307543/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:06,628 INFO [StoreOpener-d7d21b347a5059d92fcfbcd4fc307543-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:06,629 DEBUG [StoreOpener-d7d21b347a5059d92fcfbcd4fc307543-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2 2023-02-07 15:54:06,629 DEBUG [StoreOpener-d7d21b347a5059d92fcfbcd4fc307543-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2 2023-02-07 15:54:06,631 INFO [StoreOpener-d7d21b347a5059d92fcfbcd4fc307543-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d7d21b347a5059d92fcfbcd4fc307543 columnFamilyName cf2 2023-02-07 15:54:06,632 INFO [StoreOpener-d7d21b347a5059d92fcfbcd4fc307543-1] regionserver.HStore(310): Store=d7d21b347a5059d92fcfbcd4fc307543/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:06,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:06,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:06,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:06,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:06,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:54:06,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened d7d21b347a5059d92fcfbcd4fc307543; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73530588, jitterRate=0.09569114446640015}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:06,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for d7d21b347a5059d92fcfbcd4fc307543: Running coprocessor pre-open hook at 1675785246624Writing region info on filesystem at 1675785246624Initializing all the Stores at 1675785246624Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785246625 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785246625Cleaning up temporary data from old regions at 1675785246634 (+9 ms)Cleaning up detritus from prior splits at 1675785246635 (+1 ms)Running coprocessor post-open hooks at 1675785246640 (+5 ms)Region opened successfully at 1675785246640 2023-02-07 15:54:06,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543., pid=27, masterSystemTime=1675785246619 2023-02-07 15:54:06,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:06,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:06,644 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=d7d21b347a5059d92fcfbcd4fc307543, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,644 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785246644"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785246644"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785246644"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785246644"}]},"ts":"1675785246644"} 2023-02-07 15:54:06,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-07 15:54:06,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 in 179 msec 2023-02-07 15:54:06,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-07 15:54:06,654 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, ASSIGN in 341 msec 2023-02-07 15:54:06,655 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 15:54:06,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785246655"}]},"ts":"1675785246655"} 2023-02-07 15:54:06,657 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-07 15:54:06,708 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 15:54:06,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 467 msec 2023-02-07 15:54:06,747 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-07 15:54:06,750 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:51088, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-07 15:54:06,758 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:06,758 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:06,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:06,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-07 15:54:06,762 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-07 15:54:06,762 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:06,763 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-07 15:54:06,763 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-07 15:54:06,764 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,764 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:06,776 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:06,776 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,776 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:06,776 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:06,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:06,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:06,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:06,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:06,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:06,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:06,777 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,777 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:06,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:06,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:06,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:06,778 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:06,778 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:06,779 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:06,779 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:06,779 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:06,779 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:06,779 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:06,780 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:06,780 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:06,781 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:06,781 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:06,781 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:06,781 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:06,781 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:06,781 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:06,781 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:06,781 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. started... 2023-02-07 15:54:06,782 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d7d21b347a5059d92fcfbcd4fc307543 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 15:54:06,791 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,800 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/705f4720d12b407f8402e41811956243 2023-02-07 15:54:06,802 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,802 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,802 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:06,802 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,802 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:06,817 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/09deb9b171b64d2685cf8fa5680d26fa 2023-02-07 15:54:06,823 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/705f4720d12b407f8402e41811956243 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243 2023-02-07 15:54:06,829 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:54:06,830 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/09deb9b171b64d2685cf8fa5680d26fa as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa 2023-02-07 15:54:06,836 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:54:06,837 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d7d21b347a5059d92fcfbcd4fc307543 in 55ms, sequenceid=6, compaction requested=false 2023-02-07 15:54:06,837 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-07 15:54:06,837 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d7d21b347a5059d92fcfbcd4fc307543: 2023-02-07 15:54:06,837 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:06,837 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:06,837 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:06,837 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,837 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:06,837 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:06,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,850 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:06,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:06,851 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,851 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:06,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:06,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:06,852 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-07 15:54:06,853 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50461a15[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:06,853 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-07 15:54:06,853 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,865 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,865 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,865 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,865 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,865 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:06,866 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:06,865 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:06,866 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:06,865 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:06,866 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:06,866 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:06,866 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:06,866 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:06,866 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:06,875 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:06,875 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:06,876 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:06,886 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:06,886 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:06,886 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,886 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:06,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:06,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:06,886 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:06,887 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:06,887 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:06,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:06,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:06,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:06,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:06,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:06,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:06,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:06,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:06,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-07 15:54:06,891 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-07 15:54:06,891 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:06,891 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:06,891 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-07 15:54:06,891 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:06,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:06,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:06,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:06,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,903 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:06,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,903 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:06,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:06,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,904 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:06,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:06,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:06,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:06,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:06,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:06,904 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:06,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:06,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:06,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:06,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:06,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:06,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:06,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:06,905 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:06,905 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:06,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:06,906 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:06,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:06,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:06,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:06,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:06,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:06,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:06,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:06,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:06,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:06,918 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:06,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:06,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:06,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:06,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:06,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:06,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:06,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:06,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-07 15:54:06,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:06,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,918 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:06,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:06,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:06,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:06,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:06,919 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:06,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:06,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:06,920 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:06,920 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:06,920 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:06,920 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,920 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:06,920 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:06,920 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:06,920 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,020 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:07,020 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:07,020 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:07,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:07,041 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:07,041 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:07,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:07,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:07,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-07 15:54:07,044 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-07 15:54:07,044 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:07,045 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-07 15:54:07,045 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-07 15:54:07,045 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,045 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,054 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,054 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,054 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,054 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,055 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,055 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:07,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,055 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,055 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,056 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,056 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:07,056 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:07,056 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:07,056 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:07,056 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:07,056 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:07,056 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:07,056 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:07,056 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:07,056 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:07,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:07,058 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:07,058 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:07,058 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:07,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:07,058 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:07,059 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:07,059 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:07,059 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:07,059 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:07,059 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:07,060 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:07,060 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:07,060 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:07,060 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. started... 2023-02-07 15:54:07,060 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d7d21b347a5059d92fcfbcd4fc307543 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:07,071 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,071 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/233eb9b69d00434bb007481a745a83c0 2023-02-07 15:54:07,082 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,082 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,082 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:07,082 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,082 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:07,086 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/843470a3f5a342dea4ea5b43d8086dab 2023-02-07 15:54:07,091 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/233eb9b69d00434bb007481a745a83c0 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0 2023-02-07 15:54:07,097 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:54:07,098 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/843470a3f5a342dea4ea5b43d8086dab as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab 2023-02-07 15:54:07,103 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:54:07,105 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d7d21b347a5059d92fcfbcd4fc307543 in 45ms, sequenceid=19, compaction requested=false 2023-02-07 15:54:07,105 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d7d21b347a5059d92fcfbcd4fc307543: 2023-02-07 15:54:07,105 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:07,105 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:07,105 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:07,105 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,105 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:07,105 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:07,117 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,118 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:07,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:07,118 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,118 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:07,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:07,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:07,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:07,120 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-07 15:54:07,120 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@142342c6[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:07,120 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-07 15:54:07,120 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,128 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,128 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,128 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,128 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,128 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:07,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,129 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:07,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,129 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:07,129 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:07,129 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:07,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,129 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:07,129 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:07,129 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:07,129 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:07,129 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:07,168 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:07,168 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:07,168 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:07,187 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:07,187 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:07,187 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,187 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:07,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:07,187 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:07,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:07,188 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:07,188 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:07,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:07,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:07,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:07,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:07,193 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-07 15:54:07,193 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-07 15:54:07,193 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:07,193 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:07,193 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-07 15:54:07,193 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:07,202 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,202 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,202 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,202 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,202 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:07,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:07,202 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,203 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:07,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:07,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,205 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,205 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,206 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:07,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,207 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,208 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,208 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,219 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,219 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,219 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,219 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-07 15:54:07,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:07,219 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:07,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,219 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,219 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,219 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,220 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,219 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,220 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,220 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,321 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:07,321 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:07,321 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:07,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:07,342 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:07,342 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:07,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:07,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:07,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-07 15:54:07,345 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-07 15:54:07,345 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:07,346 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-07 15:54:07,346 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-07 15:54:07,347 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,347 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,421 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,421 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,421 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,421 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,422 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,422 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:07,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,422 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:07,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:07,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:07,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:07,423 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:07,423 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:07,423 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:07,423 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:07,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:07,425 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:07,426 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:07,426 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:07,425 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:07,426 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:07,426 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:07,426 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:07,433 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:07,433 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:07,433 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:07,434 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:07,433 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:07,434 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:07,433 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. started... 2023-02-07 15:54:07,434 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d7d21b347a5059d92fcfbcd4fc307543 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:07,446 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/9df53007e90f462d856925699ebb9c6c 2023-02-07 15:54:07,454 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,464 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/91ed2f39898844728474d4a882035e5d 2023-02-07 15:54:07,465 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,465 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:07,465 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,465 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,465 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:07,470 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/9df53007e90f462d856925699ebb9c6c as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c 2023-02-07 15:54:07,476 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:54:07,477 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/91ed2f39898844728474d4a882035e5d as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d 2023-02-07 15:54:07,482 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:54:07,483 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d7d21b347a5059d92fcfbcd4fc307543 in 49ms, sequenceid=32, compaction requested=true 2023-02-07 15:54:07,483 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d7d21b347a5059d92fcfbcd4fc307543: 2023-02-07 15:54:07,483 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:07,483 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:07,483 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:07,483 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,483 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:07,483 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:07,496 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,496 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:07,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:07,497 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,497 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:07,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:07,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:07,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:07,499 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-07 15:54:07,499 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d3c3555[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:07,499 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-07 15:54:07,499 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,507 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,507 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,507 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,507 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:07,507 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:07,507 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,508 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:07,507 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:07,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,508 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:07,508 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:07,508 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:07,508 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:07,508 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:07,508 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:07,517 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:07,518 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:07,518 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:07,528 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:07,528 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:07,528 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,528 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:07,529 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:07,528 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:07,529 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:07,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:07,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:07,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:07,533 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-07 15:54:07,533 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-07 15:54:07,534 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:07,534 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:07,534 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-07 15:54:07,534 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:07,545 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,545 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:07,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:07,545 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,545 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,545 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,545 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,545 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:07,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,546 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,547 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:07,547 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:07,548 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,549 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,549 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-07 15:54:07,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:07,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,560 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:07,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,562 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:07,562 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:07,561 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,562 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,562 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,562 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,562 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,562 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,562 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,562 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,662 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:07,662 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:07,662 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:07,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:07,683 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:07,683 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:07,684 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:07,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:07,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-07 15:54:07,686 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-07 15:54:07,686 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:07,687 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-07 15:54:07,687 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-07 15:54:07,688 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,688 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,758 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,758 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,758 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,758 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,759 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,759 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:07,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:07,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:07,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:07,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,760 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:07,760 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:07,760 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:07,760 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:07,760 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:07,760 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:07,760 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:07,763 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:07,763 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:07,763 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:07,763 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:07,764 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:07,763 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:07,764 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:07,764 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:07,764 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:07,764 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:07,766 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:07,764 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:07,766 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:07,766 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:07,766 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. started... 2023-02-07 15:54:07,766 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:07,766 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:07,766 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:07,766 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:07,766 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d7d21b347a5059d92fcfbcd4fc307543 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:07,777 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/2227cf2356db4b0db8439d6d6653f63a 2023-02-07 15:54:07,787 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,794 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/95069490efc94ba498dab60b38c4f21f 2023-02-07 15:54:07,797 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,797 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,797 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:07,798 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,798 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:07,799 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/2227cf2356db4b0db8439d6d6653f63a as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a 2023-02-07 15:54:07,805 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:54:07,806 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/95069490efc94ba498dab60b38c4f21f as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f 2023-02-07 15:54:07,811 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:54:07,813 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d7d21b347a5059d92fcfbcd4fc307543 in 47ms, sequenceid=45, compaction requested=true 2023-02-07 15:54:07,813 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d7d21b347a5059d92fcfbcd4fc307543: 2023-02-07 15:54:07,813 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:07,813 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:07,813 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:07,813 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,813 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:07,813 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:07,823 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,823 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:07,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:07,823 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,823 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:07,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:07,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:07,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:07,826 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-07 15:54:07,826 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@451dc450[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:07,826 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-07 15:54:07,826 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,833 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,833 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,833 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,834 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:07,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,834 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,834 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:07,834 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:07,834 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:07,834 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:07,834 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:07,834 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:07,834 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:07,834 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:07,834 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:07,844 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:07,844 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:07,844 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:07,854 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:07,854 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:07,855 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:07,855 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:07,854 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,855 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:07,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:07,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:07,856 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:07,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:07,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:07,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:07,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:07,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-07 15:54:07,862 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-07 15:54:07,862 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:07,862 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:07,862 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-07 15:54:07,862 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:07,871 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,871 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,871 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,871 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:07,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:07,871 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,872 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,872 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:07,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,872 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:07,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,873 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,873 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:07,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:07,874 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,875 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,875 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,886 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,886 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,886 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,886 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:07,886 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,886 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,886 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,887 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,887 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,886 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:07,887 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:07,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,886 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-07 15:54:07,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,886 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:07,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:07,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:07,887 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:07,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:07,887 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,887 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:07,887 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:07,887 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:07,887 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:07,888 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:07,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:07,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:07,888 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:07,988 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:07,988 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:07,988 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:07,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:08,010 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:08,010 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:08,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:08,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:08,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-07 15:54:08,013 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-07 15:54:08,013 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:08,014 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-07 15:54:08,014 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-07 15:54:08,014 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,014 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,023 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:08,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:08,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:08,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:08,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:08,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:08,023 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,023 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:08,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:08,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:08,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,024 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:08,025 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:08,025 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:08,025 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:08,025 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:08,025 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:08,025 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:08,025 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:08,025 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:08,025 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:08,025 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:08,025 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:08,025 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:08,025 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:08,027 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:08,027 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:08,027 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:08,027 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:08,027 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:08,028 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:08,028 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:08,029 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:08,029 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:08,029 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:08,029 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. started... 2023-02-07 15:54:08,029 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d7d21b347a5059d92fcfbcd4fc307543 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:08,033 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,044 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,044 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:08,044 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,044 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,044 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:08,046 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/6939e3c25b0e4c8b991054375ebf8403 2023-02-07 15:54:08,063 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/f6845efcabe54979989222bb24ea9b3c 2023-02-07 15:54:08,070 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/6939e3c25b0e4c8b991054375ebf8403 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403 2023-02-07 15:54:08,077 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:54:08,078 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/f6845efcabe54979989222bb24ea9b3c as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c 2023-02-07 15:54:08,085 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:54:08,087 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d7d21b347a5059d92fcfbcd4fc307543 in 58ms, sequenceid=58, compaction requested=true 2023-02-07 15:54:08,087 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d7d21b347a5059d92fcfbcd4fc307543: 2023-02-07 15:54:08,087 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:08,087 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:08,087 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:08,087 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,087 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:08,087 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:08,117 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,117 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:08,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:08,118 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:08,118 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:08,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:08,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:08,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:08,120 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-07 15:54:08,120 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25ebf854[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:08,120 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-07 15:54:08,120 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,233 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,233 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,233 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,234 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:08,234 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:08,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,234 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,234 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:08,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,234 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:08,234 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:08,234 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:08,234 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:08,234 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:08,234 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:08,234 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:08,337 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:08,337 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:08,337 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:08,337 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:08,337 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:08,337 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:08,360 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:08,360 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:08,360 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,360 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:08,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:08,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:08,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:08,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:08,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:08,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:08,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:08,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:08,364 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-07 15:54:08,364 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-07 15:54:08,364 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:08,364 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:08,364 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-07 15:54:08,364 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:08,401 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,401 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,402 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:08,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,401 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,401 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:08,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,402 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,402 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:08,402 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:08,403 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:08,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,403 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:08,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,404 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:08,405 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:08,405 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:08,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:08,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:08,423 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:08,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:08,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:08,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-07 15:54:08,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:08,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:08,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:08,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:08,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:08,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:08,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,424 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:08,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:08,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:08,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:08,424 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,424 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:08,424 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:08,424 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:08,424 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:08,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:08,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:08,424 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,524 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:08,524 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:08,524 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:08,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:08,546 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:08,546 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:08,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:08,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:08,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-07 15:54:08,549 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-07 15:54:08,549 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:08,549 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-07 15:54:08,550 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-07 15:54:08,550 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,550 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:08,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:08,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:08,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:08,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:08,560 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:08,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,560 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,561 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 15:54:08,561 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:08,561 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:08,561 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:08,561 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:08,562 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:08,561 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:08,562 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:08,562 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:08,562 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:08,562 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. started... 2023-02-07 15:54:08,562 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d7d21b347a5059d92fcfbcd4fc307543 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:08,570 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,573 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/33d15303282549ae8de8a01a5cd0899d 2023-02-07 15:54:08,580 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,581 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,581 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:08,581 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,581 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:08,587 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/20ed63de60fc4c39a10df02bb655da27 2023-02-07 15:54:08,592 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/33d15303282549ae8de8a01a5cd0899d as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d 2023-02-07 15:54:08,598 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:54:08,599 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/20ed63de60fc4c39a10df02bb655da27 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27 2023-02-07 15:54:08,604 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:54:08,605 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d7d21b347a5059d92fcfbcd4fc307543 in 43ms, sequenceid=71, compaction requested=true 2023-02-07 15:54:08,606 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d7d21b347a5059d92fcfbcd4fc307543: 2023-02-07 15:54:08,606 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:08,606 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:08,606 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:08,606 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,606 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 15:54:08,606 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 15:54:08,619 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,619 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:08,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:08,619 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,619 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:08,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:08,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:08,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:08,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:08,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-07 15:54:08,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2bd0ab5d[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:08,621 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-07 15:54:08,621 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,633 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,633 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,633 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,634 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,634 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:08,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,634 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:08,634 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:08,634 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:08,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,634 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:08,634 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:08,634 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 15:54:08,634 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 15:54:08,634 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:08,634 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:08,644 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,644 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:08,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:08,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:08,644 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:08,644 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:08,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:08,733 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:08,733 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 15:54:08,733 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:08,733 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:08,733 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:08,733 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 15:54:08,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:08,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:08,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:08,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:08,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:08,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-07 15:54:08,736 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-07 15:54:08,737 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:08,737 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:08,737 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-07 15:54:08,737 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:08,766 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,766 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,766 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,766 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:08,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,766 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:08,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:08,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,766 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:08,767 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,766 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:08,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:08,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:08,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:08,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:08,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:08,767 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:08,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:08,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:08,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,768 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:08,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:08,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,769 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:08,769 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 15:54:08,770 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,781 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:08,781 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:08,781 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:08,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:08,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:08,781 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:08,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:08,781 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,781 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:08,781 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:08,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:08,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-07 15:54:08,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:08,781 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:08,781 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:08,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:08,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:08,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:08,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:08,781 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,782 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:08,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:08,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:08,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:08,782 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:08,782 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,782 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 15:54:08,782 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:08,782 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,782 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:08,782 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:08,782 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 15:54:08,782 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 15:54:08,882 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:08,883 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:08,883 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:08,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:09,148 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-07 15:54:09,150 DEBUG [Listener at localhost.localdomain/41965] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:09,151 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,152 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,152 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,152 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,153 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:09,153 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:09,153 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:09,401 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,402 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:09,402 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:09,403 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:09,405 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-07 15:54:09,406 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:09,406 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:09,406 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:09,406 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:09,407 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:09,410 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-07 15:54:09,412 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:09,412 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:09,412 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:09,413 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:09,413 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:09,413 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:09,414 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:09,414 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:09,414 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:09,415 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,415 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:09,415 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,415 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,415 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,416 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-07 15:54:09,416 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:09,417 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:09,422 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785279422 2023-02-07 15:54:09,424 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,425 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:09,429 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:09,430 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d7d21b347a5059d92fcfbcd4fc307543, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:09,430 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:09,430 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785309430 2023-02-07 15:54:09,430 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785369430 2023-02-07 15:54:09,431 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-07 15:54:09,431 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:09,434 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:09,434 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:09,434 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:09,681 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,681 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:09,681 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:09,681 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:09,682 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:09,682 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:09,682 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:09,682 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:09,682 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:09,683 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:09,685 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-07 15:54:09,686 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:09,686 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:09,686 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:09,687 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:09,687 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:09,687 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:09,688 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:09,688 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:09,688 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:09,689 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,689 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:09,689 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,689 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,689 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,689 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 15:54:09,690 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:09,690 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:09,691 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785279691 2023-02-07 15:54:09,692 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,692 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:09,694 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:09,695 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d7d21b347a5059d92fcfbcd4fc307543, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:09,695 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:09,695 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785309695 2023-02-07 15:54:09,695 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785369695 2023-02-07 15:54:09,695 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:09,695 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:09,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN}] 2023-02-07 15:54:09,698 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:09,698 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:09,698 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:09,945 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,945 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:09,945 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:09,946 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:09,946 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:09,947 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:09,947 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:09,947 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:09,947 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:09,948 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:09,950 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-07 15:54:09,950 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:09,951 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:09,951 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:09,951 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:09,951 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:09,952 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:09,952 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:09,952 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:09,953 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:09,953 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,953 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:09,953 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,954 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,954 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,954 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 15:54:09,955 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:09,955 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:09,955 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785279955 2023-02-07 15:54:09,956 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:09,956 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:09,959 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:09,959 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d7d21b347a5059d92fcfbcd4fc307543, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:09,959 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:09,959 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785309959 2023-02-07 15:54:09,959 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785369959 2023-02-07 15:54:09,959 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:09,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN}] 2023-02-07 15:54:09,960 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:09,962 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:09,963 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=d7d21b347a5059d92fcfbcd4fc307543, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:09,963 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785249963"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785249963"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785249963"}]},"ts":"1675785249963"} 2023-02-07 15:54:09,965 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:09,965 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:09,965 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:09,965 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:09,965 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:10,081 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 15:54:10,210 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,210 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:10,210 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:10,210 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:10,210 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:10,210 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:10,210 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:10,210 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:10,211 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:10,211 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:10,213 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 15:54:10,213 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-07 15:54:10,214 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:10,214 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:10,214 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:10,214 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:10,215 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:10,215 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:10,215 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:10,215 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:10,216 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:10,217 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,217 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:10,217 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:10,217 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,217 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,217 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,218 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:10,218 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:10,218 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 15:54:10,219 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:10,219 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:10,219 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543 to restore RIT 2023-02-07 15:54:10,220 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785280220 2023-02-07 15:54:10,220 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:10,220 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:10,223 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:10,223 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d7d21b347a5059d92fcfbcd4fc307543, regionState=CLOSING, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:10,223 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:10,223 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785310223 2023-02-07 15:54:10,223 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785370223 2023-02-07 15:54:10,223 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:10,224 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=d7d21b347a5059d92fcfbcd4fc307543, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:10,224 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785250223"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785250223"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785250223"}]},"ts":"1675785250223"} 2023-02-07 15:54:10,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:10,226 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:10,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:10,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing d7d21b347a5059d92fcfbcd4fc307543, disabling compactions & flushes 2023-02-07 15:54:10,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:10,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:10,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. after waiting 0 ms 2023-02-07 15:54:10,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:10,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing d7d21b347a5059d92fcfbcd4fc307543 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 15:54:10,390 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/8bb480d936e243fc9e70fe75a2df5999 2023-02-07 15:54:10,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/fd4e3f8164354994af586cdf89b19e29 2023-02-07 15:54:10,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf1/8bb480d936e243fc9e70fe75a2df5999 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999 2023-02-07 15:54:10,426 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:54:10,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/.tmp/cf2/fd4e3f8164354994af586cdf89b19e29 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29 2023-02-07 15:54:10,433 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:54:10,434 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d7d21b347a5059d92fcfbcd4fc307543 in 55ms, sequenceid=82, compaction requested=true 2023-02-07 15:54:10,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 15:54:10,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:10,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. 2023-02-07 15:54:10,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for d7d21b347a5059d92fcfbcd4fc307543: Waiting for close lock at 1675785250379Running coprocessor pre-close hooks at 1675785250379Disabling compacts and flushes for region at 1675785250379Disabling writes for close at 1675785250379Obtaining lock to block concurrent updates at 1675785250379Preparing flush snapshotting stores in d7d21b347a5059d92fcfbcd4fc307543 at 1675785250379Finished memstore snapshotting testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675785250379Flushing stores of testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. at 1675785250380 (+1 ms)Flushing d7d21b347a5059d92fcfbcd4fc307543/cf1: creating writer at 1675785250381 (+1 ms)Flushing d7d21b347a5059d92fcfbcd4fc307543/cf1: appending metadata at 1675785250383 (+2 ms)Flushing d7d21b347a5059d92fcfbcd4fc307543/cf1: closing flushed file at 1675785250383Flushing d7d21b347a5059d92fcfbcd4fc307543/cf2: creating writer at 1675785250396 (+13 ms)Flushing d7d21b347a5059d92fcfbcd4fc307543/cf2: appending metadata at 1675785250399 (+3 ms)Flushing d7d21b347a5059d92fcfbcd4fc307543/cf2: closing flushed file at 1675785250399Flushing d7d21b347a5059d92fcfbcd4fc307543/cf1: reopening flushed file at 1675785250421 (+22 ms)Flushing d7d21b347a5059d92fcfbcd4fc307543/cf2: reopening flushed file at 1675785250427 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d7d21b347a5059d92fcfbcd4fc307543 in 55ms, sequenceid=82, compaction requested=true at 1675785250434 (+7 ms)Writing region close event to WAL at 1675785250448 (+14 ms)Running coprocessor post-close hooks at 1675785250452 (+4 ms)Closed at 1675785250453 (+1 ms) 2023-02-07 15:54:10,456 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:10,456 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=d7d21b347a5059d92fcfbcd4fc307543, regionState=CLOSED 2023-02-07 15:54:10,456 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785250456"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785250456"}]},"ts":"1675785250456"} 2023-02-07 15:54:10,458 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:10,458 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:10,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:10,458 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:10,473 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,474 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:10,474 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:10,474 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:10,474 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:10,474 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:10,474 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:10,474 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:10,474 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:10,475 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:10,476 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 15:54:10,476 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-07 15:54:10,477 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-07 15:54:10,477 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:10,477 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:10,477 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:10,477 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:10,478 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:10,478 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:10,478 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:10,478 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:10,479 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:10,479 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,479 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:10,479 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:10,479 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:10,480 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,480 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,480 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,480 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:10,480 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:10,480 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:10,480 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 15:54:10,481 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:10,481 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:10,481 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543 to restore RIT 2023-02-07 15:54:10,481 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785280481 2023-02-07 15:54:10,482 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:10,482 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:10,485 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:10,485 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d7d21b347a5059d92fcfbcd4fc307543, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:10,485 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:10,485 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785310485 2023-02-07 15:54:10,485 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785370485 2023-02-07 15:54:10,486 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:10,486 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=d7d21b347a5059d92fcfbcd4fc307543, regionState=CLOSED, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:10,486 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785250486"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785250486"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785250486"}]},"ts":"1675785250486"} 2023-02-07 15:54:10,488 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:10,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-07 15:54:10,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 in 262 msec 2023-02-07 15:54:10,491 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:10,491 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:10,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:10,492 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:10,736 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,736 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:10,736 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:10,736 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:10,736 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:10,736 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:10,736 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:10,737 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:10,737 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:10,737 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:10,739 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 15:54:10,739 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-07 15:54:10,739 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:10,740 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:10,740 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:10,740 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:10,741 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:10,741 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:10,742 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:10,742 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:10,742 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:10,742 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:10,743 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,743 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:10,743 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:10,743 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:10,744 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,744 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,744 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,744 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:10,744 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:10,744 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 15:54:10,745 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:10,745 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:10,746 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543 to restore RIT 2023-02-07 15:54:10,746 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785280746 2023-02-07 15:54:10,747 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:10,747 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:10,749 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:10,750 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d7d21b347a5059d92fcfbcd4fc307543, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:10,750 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:10,750 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785310750 2023-02-07 15:54:10,750 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785370750 2023-02-07 15:54:10,750 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:10,750 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:10,752 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:10,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-07 15:54:10,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN in 792 msec 2023-02-07 15:54:10,759 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:10,759 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:10,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:10,759 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:11,000 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,000 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:11,000 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:11,000 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:11,001 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:11,001 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:11,001 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:11,001 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:11,001 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:11,002 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:11,003 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 15:54:11,003 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:54:11,003 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:11,003 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:11,004 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:11,004 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:11,004 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:11,004 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:11,005 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:11,005 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:11,005 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:11,005 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:11,006 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,006 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:11,006 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:11,006 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:11,006 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:11,006 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:11,006 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 15:54:11,007 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:11,007 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:11,008 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785281008 2023-02-07 15:54:11,009 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,009 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:11,011 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:11,011 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d7d21b347a5059d92fcfbcd4fc307543, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:11,012 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:11,012 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785311012 2023-02-07 15:54:11,012 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785371012 2023-02-07 15:54:11,012 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:54:11,012 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,012 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,012 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,017 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:11,027 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=d7d21b347a5059d92fcfbcd4fc307543, threads=14 2023-02-07 15:54:11,028 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,028 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,028 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,028 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,029 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,029 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,029 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,029 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,029 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,029 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,029 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,030 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,030 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,033 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,201 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,213 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,253 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,256 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,256 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,257 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,259 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,259 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,260 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,263 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,264 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,544 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,582 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,664 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,664 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region d7d21b347a5059d92fcfbcd4fc307543 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 15:54:11,707 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:11,707 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:11,707 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:11,762 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,762 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:11,762 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:11,762 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:11,763 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:11,763 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:11,763 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:11,763 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:11,763 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:11,764 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:11,765 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-07 15:54:11,765 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:54:11,765 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:11,766 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:11,766 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:11,766 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:11,767 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:11,767 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:11,767 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:11,768 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:11,768 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:11,768 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:11,769 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,769 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:11,769 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:11,769 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:11,769 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,769 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,770 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,770 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:11,770 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:11,770 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 15:54:11,770 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:11,771 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:11,772 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785281772 2023-02-07 15:54:11,774 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:11,774 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:11,776 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:11,776 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d7d21b347a5059d92fcfbcd4fc307543, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:11,777 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:11,777 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785311777 2023-02-07 15:54:11,777 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785371777 2023-02-07 15:54:11,777 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:11,787 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=d7d21b347a5059d92fcfbcd4fc307543, threads=14 2023-02-07 15:54:11,788 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,788 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,789 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,789 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,789 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,793 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,793 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,793 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,793 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,793 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,793 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,794 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,794 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,794 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:11,989 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,013 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,013 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,045 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,053 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,061 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,061 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,062 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,063 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,063 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,063 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,064 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,064 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27 for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,065 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa for region: d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:12,066 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region d7d21b347a5059d92fcfbcd4fc307543 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 15:54:12,109 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:12,115 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 15:54:12,117 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 15:54:12,117 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:12,117 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:12,117 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:12,277 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,277 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:12,277 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:12,277 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:12,278 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:12,278 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:12,278 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:12,278 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:12,278 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:12,279 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:12,280 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-07 15:54:12,280 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:54:12,280 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:12,280 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:12,280 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:12,280 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:12,281 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:12,281 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:12,281 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:12,281 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:12,281 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:12,282 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:12,282 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,282 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:12,282 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:12,282 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:12,282 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,282 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,283 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,283 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:12,283 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:12,283 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 15:54:12,284 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:12,284 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:12,284 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785282284 2023-02-07 15:54:12,286 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,286 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:12,288 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:12,289 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d7d21b347a5059d92fcfbcd4fc307543, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:12,289 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:12,289 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785312289 2023-02-07 15:54:12,289 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785372289 2023-02-07 15:54:12,289 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:12,292 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:12,294 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:12,294 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:12,294 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:12,539 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:12,539 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:12,539 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:12,539 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:12,540 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:12,541 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:12,541 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:12,541 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:12,541 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:12,542 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:12,542 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-07 15:54:12,543 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:54:12,543 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:12,543 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:12,543 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:12,543 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:12,543 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:12,544 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:12,544 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:12,544 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:12,545 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:12,545 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:12,546 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,546 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN 2023-02-07 15:54:12,546 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:12,546 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:12,546 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,546 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,546 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,547 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d7d21b347a5059d92fcfbcd4fc307543, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:12,547 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure d7d21b347a5059d92fcfbcd4fc307543, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:12,547 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 15:54:12,547 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:12,548 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:12,557 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785282557 2023-02-07 15:54:12,558 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:12,558 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:12,561 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:12,561 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d7d21b347a5059d92fcfbcd4fc307543, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:12,561 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:12,561 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785312561 2023-02-07 15:54:12,561 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785372561 2023-02-07 15:54:12,561 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:12,566 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785252566"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675785252566"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675785252566"}]},"ts":"1675785252566"} 2023-02-07 15:54:12,566 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785252566"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785252566"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785252566"}]},"ts":"1675785252566"} 2023-02-07 15:54:12,566 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785252566"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785252566"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785252566"}]},"ts":"1675785252566"} 2023-02-07 15:54:12,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c9f79cf0473a30691a3c2d31c830aa7, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fadaaa28c739d279eb5c9fd9b2befe27, ASSIGN}] 2023-02-07 15:54:12,575 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fadaaa28c739d279eb5c9fd9b2befe27, ASSIGN 2023-02-07 15:54:12,577 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fadaaa28c739d279eb5c9fd9b2befe27, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:54:12,578 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c9f79cf0473a30691a3c2d31c830aa7, ASSIGN 2023-02-07 15:54:12,579 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c9f79cf0473a30691a3c2d31c830aa7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:54:12,727 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 15:54:12,728 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=fadaaa28c739d279eb5c9fd9b2befe27, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:12,729 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785252728"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785252728"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785252728"}]},"ts":"1675785252728"} 2023-02-07 15:54:12,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure fadaaa28c739d279eb5c9fd9b2befe27, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:12,734 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=0c9f79cf0473a30691a3c2d31c830aa7, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:12,735 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785252734"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785252734"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785252734"}]},"ts":"1675785252734"} 2023-02-07 15:54:12,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 0c9f79cf0473a30691a3c2d31c830aa7, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:12,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7. 2023-02-07 15:54:12,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0c9f79cf0473a30691a3c2d31c830aa7, NAME => 'testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.', STARTKEY => '', ENDKEY => '41'} 2023-02-07 15:54:12,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:12,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 0c9f79cf0473a30691a3c2d31c830aa7 2023-02-07 15:54:12,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:12,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 0c9f79cf0473a30691a3c2d31c830aa7 2023-02-07 15:54:12,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 0c9f79cf0473a30691a3c2d31c830aa7 2023-02-07 15:54:12,889 INFO [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0c9f79cf0473a30691a3c2d31c830aa7 2023-02-07 15:54:12,890 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1 2023-02-07 15:54:12,890 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1 2023-02-07 15:54:12,890 INFO [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0c9f79cf0473a30691a3c2d31c830aa7 columnFamilyName cf1 2023-02-07 15:54:12,909 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a-bottom 2023-02-07 15:54:12,915 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0-bottom 2023-02-07 15:54:12,921 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d-bottom 2023-02-07 15:54:12,927 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403-bottom 2023-02-07 15:54:12,933 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243-bottom 2023-02-07 15:54:12,939 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999-bottom 2023-02-07 15:54:12,946 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c-bottom 2023-02-07 15:54:12,946 INFO [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(310): Store=0c9f79cf0473a30691a3c2d31c830aa7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:12,946 INFO [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0c9f79cf0473a30691a3c2d31c830aa7 2023-02-07 15:54:12,947 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2 2023-02-07 15:54:12,947 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2 2023-02-07 15:54:12,948 INFO [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0c9f79cf0473a30691a3c2d31c830aa7 columnFamilyName cf2 2023-02-07 15:54:12,968 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa-bottom 2023-02-07 15:54:12,975 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27-bottom 2023-02-07 15:54:12,982 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab-bottom 2023-02-07 15:54:12,988 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d-bottom 2023-02-07 15:54:12,995 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f-bottom 2023-02-07 15:54:13,001 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c-bottom 2023-02-07 15:54:13,007 DEBUG [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29-bottom 2023-02-07 15:54:13,008 INFO [StoreOpener-0c9f79cf0473a30691a3c2d31c830aa7-1] regionserver.HStore(310): Store=0c9f79cf0473a30691a3c2d31c830aa7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:13,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7 2023-02-07 15:54:13,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7 2023-02-07 15:54:13,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:13,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 0c9f79cf0473a30691a3c2d31c830aa7 2023-02-07 15:54:13,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 0c9f79cf0473a30691a3c2d31c830aa7; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64434632, jitterRate=-0.039849162101745605}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:13,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 0c9f79cf0473a30691a3c2d31c830aa7: Running coprocessor pre-open hook at 1675785252888Writing region info on filesystem at 1675785252888Initializing all the Stores at 1675785252889 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785252889Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785252889Cleaning up temporary data from old regions at 1675785253011 (+122 ms)Cleaning up detritus from prior splits at 1675785253011Running coprocessor post-open hooks at 1675785253014 (+3 ms)Region opened successfully at 1675785253014 2023-02-07 15:54:13,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7., pid=34, masterSystemTime=1675785252883 2023-02-07 15:54:13,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:13,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:54:13,017 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:54:13,017 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:54:13,017 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7. 2023-02-07 15:54:13,017 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7. 2023-02-07 15:54:13,017 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1912): 0c9f79cf0473a30691a3c2d31c830aa7/cf1 is initiating minor compaction (all files) 2023-02-07 15:54:13,017 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1912): 0c9f79cf0473a30691a3c2d31c830aa7/cf2 is initiating minor compaction (all files) 2023-02-07 15:54:13,017 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0c9f79cf0473a30691a3c2d31c830aa7/cf1 in testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7. 2023-02-07 15:54:13,017 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 0c9f79cf0473a30691a3c2d31c830aa7/cf2 in testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7. 2023-02-07 15:54:13,018 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29-bottom] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/.tmp, totalSize=34.9 K 2023-02-07 15:54:13,018 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999-bottom] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/.tmp, totalSize=34.9 K 2023-02-07 15:54:13,019 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785246750 2023-02-07 15:54:13,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7. 2023-02-07 15:54:13,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7. 2023-02-07 15:54:13,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27. 2023-02-07 15:54:13,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fadaaa28c739d279eb5c9fd9b2befe27, NAME => 'testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.', STARTKEY => '41', ENDKEY => ''} 2023-02-07 15:54:13,019 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785246750 2023-02-07 15:54:13,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:13,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:13,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:13,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:13,020 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785247022 2023-02-07 15:54:13,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:13,020 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=0c9f79cf0473a30691a3c2d31c830aa7, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:13,020 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785247022 2023-02-07 15:54:13,021 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785253020"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785253020"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785253020"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785253020"}]},"ts":"1675785253020"} 2023-02-07 15:54:13,021 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785247323 2023-02-07 15:54:13,021 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785247323 2023-02-07 15:54:13,022 INFO [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:13,022 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785247664 2023-02-07 15:54:13,022 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785247664 2023-02-07 15:54:13,023 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785247990 2023-02-07 15:54:13,023 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785247990 2023-02-07 15:54:13,023 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1 2023-02-07 15:54:13,023 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1 2023-02-07 15:54:13,024 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785248526 2023-02-07 15:54:13,024 INFO [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fadaaa28c739d279eb5c9fd9b2befe27 columnFamilyName cf1 2023-02-07 15:54:13,025 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675785248885 2023-02-07 15:54:13,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-07 15:54:13,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 0c9f79cf0473a30691a3c2d31c830aa7, server=jenkins-hbase12.apache.org,38629,1675785236797 in 287 msec 2023-02-07 15:54:13,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c9f79cf0473a30691a3c2d31c830aa7, ASSIGN in 454 msec 2023-02-07 15:54:13,033 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785248526 2023-02-07 15:54:13,033 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675785248885 2023-02-07 15:54:13,045 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] throttle.PressureAwareThroughputController(145): 0c9f79cf0473a30691a3c2d31c830aa7#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:13,048 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a-top 2023-02-07 15:54:13,055 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0-top 2023-02-07 15:54:13,059 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d-top 2023-02-07 15:54:13,062 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0c9f79cf0473a30691a3c2d31c830aa7#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:13,066 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403-top 2023-02-07 15:54:13,071 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243-top 2023-02-07 15:54:13,079 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999-top 2023-02-07 15:54:13,090 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c-top 2023-02-07 15:54:13,092 INFO [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(310): Store=fadaaa28c739d279eb5c9fd9b2befe27/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:13,092 INFO [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:13,093 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2 2023-02-07 15:54:13,093 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2 2023-02-07 15:54:13,094 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/.tmp/cf2/1b3b017ee69f45d38b9ba9d3ae825891 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/1b3b017ee69f45d38b9ba9d3ae825891 2023-02-07 15:54:13,094 INFO [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fadaaa28c739d279eb5c9fd9b2befe27 columnFamilyName cf2 2023-02-07 15:54:13,099 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/.tmp/cf1/8968ed0b7cff48aa8de66bad06c0dd7f as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/8968ed0b7cff48aa8de66bad06c0dd7f 2023-02-07 15:54:13,102 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0c9f79cf0473a30691a3c2d31c830aa7/cf2 of 0c9f79cf0473a30691a3c2d31c830aa7 into 1b3b017ee69f45d38b9ba9d3ae825891(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:13,102 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0c9f79cf0473a30691a3c2d31c830aa7: 2023-02-07 15:54:13,102 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7., storeName=0c9f79cf0473a30691a3c2d31c830aa7/cf2, priority=9, startTime=1675785253016; duration=0sec 2023-02-07 15:54:13,102 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:13,107 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0c9f79cf0473a30691a3c2d31c830aa7/cf1 of 0c9f79cf0473a30691a3c2d31c830aa7 into 8968ed0b7cff48aa8de66bad06c0dd7f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:13,107 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0c9f79cf0473a30691a3c2d31c830aa7: 2023-02-07 15:54:13,107 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7., storeName=0c9f79cf0473a30691a3c2d31c830aa7/cf1, priority=9, startTime=1675785253015; duration=0sec 2023-02-07 15:54:13,107 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:13,113 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa-top 2023-02-07 15:54:13,117 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27-top 2023-02-07 15:54:13,121 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab-top 2023-02-07 15:54:13,126 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d-top 2023-02-07 15:54:13,131 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f-top 2023-02-07 15:54:13,137 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c-top 2023-02-07 15:54:13,141 DEBUG [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29-top 2023-02-07 15:54:13,141 INFO [StoreOpener-fadaaa28c739d279eb5c9fd9b2befe27-1] regionserver.HStore(310): Store=fadaaa28c739d279eb5c9fd9b2befe27/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:13,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:13,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:13,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:13,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:13,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened fadaaa28c739d279eb5c9fd9b2befe27; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58975976, jitterRate=-0.12118947505950928}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:13,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for fadaaa28c739d279eb5c9fd9b2befe27: Running coprocessor pre-open hook at 1675785253021Writing region info on filesystem at 1675785253021Initializing all the Stores at 1675785253022 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785253022Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785253022Cleaning up temporary data from old regions at 1675785253144 (+122 ms)Cleaning up detritus from prior splits at 1675785253144Running coprocessor post-open hooks at 1675785253147 (+3 ms)Region opened successfully at 1675785253147 2023-02-07 15:54:13,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27., pid=33, masterSystemTime=1675785252883 2023-02-07 15:54:13,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:13,149 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:54:13,149 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:54:13,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:54:13,149 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27. 2023-02-07 15:54:13,149 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27. 2023-02-07 15:54:13,149 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1912): fadaaa28c739d279eb5c9fd9b2befe27/cf2 is initiating minor compaction (all files) 2023-02-07 15:54:13,149 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1912): fadaaa28c739d279eb5c9fd9b2befe27/cf1 is initiating minor compaction (all files) 2023-02-07 15:54:13,149 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2258): Starting compaction of fadaaa28c739d279eb5c9fd9b2befe27/cf2 in testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27. 2023-02-07 15:54:13,149 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of fadaaa28c739d279eb5c9fd9b2befe27/cf1 in testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27. 2023-02-07 15:54:13,150 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999-top] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/.tmp, totalSize=34.9 K 2023-02-07 15:54:13,150 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29-top] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/.tmp, totalSize=34.9 K 2023-02-07 15:54:13,150 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675785246750 2023-02-07 15:54:13,150 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675785246750 2023-02-07 15:54:13,151 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675785247022 2023-02-07 15:54:13,151 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675785247022 2023-02-07 15:54:13,151 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27. 2023-02-07 15:54:13,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27. 2023-02-07 15:54:13,151 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675785247323 2023-02-07 15:54:13,151 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675785247323 2023-02-07 15:54:13,152 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=fadaaa28c739d279eb5c9fd9b2befe27, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:13,152 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675785247664 2023-02-07 15:54:13,152 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675785247664 2023-02-07 15:54:13,152 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785253152"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785253152"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785253152"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785253152"}]},"ts":"1675785253152"} 2023-02-07 15:54:13,152 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675785247990 2023-02-07 15:54:13,152 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675785247990 2023-02-07 15:54:13,152 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675785248526 2023-02-07 15:54:13,153 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675785248526 2023-02-07 15:54:13,153 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675785248885 2023-02-07 15:54:13,153 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675785248885 2023-02-07 15:54:13,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-07 15:54:13,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure fadaaa28c739d279eb5c9fd9b2befe27, server=jenkins-hbase12.apache.org,38629,1675785236797 in 424 msec 2023-02-07 15:54:13,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-02-07 15:54:13,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fadaaa28c739d279eb5c9fd9b2befe27, ASSIGN in 585 msec 2023-02-07 15:54:13,162 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 in 4.0100 sec 2023-02-07 15:54:13,170 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] throttle.PressureAwareThroughputController(145): fadaaa28c739d279eb5c9fd9b2befe27#cf1#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:13,173 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] throttle.PressureAwareThroughputController(145): fadaaa28c739d279eb5c9fd9b2befe27#cf2#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:13,198 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/.tmp/cf2/393a5deb0f90415080fec81cffe4c5e6 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/393a5deb0f90415080fec81cffe4c5e6 2023-02-07 15:54:13,203 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/.tmp/cf1/fa5eb71d533d4d66803839f5ca7ff24e as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/fa5eb71d533d4d66803839f5ca7ff24e 2023-02-07 15:54:13,205 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in fadaaa28c739d279eb5c9fd9b2befe27/cf2 of fadaaa28c739d279eb5c9fd9b2befe27 into 393a5deb0f90415080fec81cffe4c5e6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:13,205 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for fadaaa28c739d279eb5c9fd9b2befe27: 2023-02-07 15:54:13,205 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27., storeName=fadaaa28c739d279eb5c9fd9b2befe27/cf2, priority=9, startTime=1675785253149; duration=0sec 2023-02-07 15:54:13,205 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:13,209 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in fadaaa28c739d279eb5c9fd9b2befe27/cf1 of fadaaa28c739d279eb5c9fd9b2befe27 into fa5eb71d533d4d66803839f5ca7ff24e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:13,209 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for fadaaa28c739d279eb5c9fd9b2befe27: 2023-02-07 15:54:13,209 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27., storeName=fadaaa28c739d279eb5c9fd9b2befe27/cf1, priority=9, startTime=1675785253148; duration=0sec 2023-02-07 15:54:13,209 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:13,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38629] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 136.243.104.168:51088 deadline: 1675785313313, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543. is not online on jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:13,427 DEBUG [Listener at localhost.localdomain/41965] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 15:54:13,436 INFO [Listener at localhost.localdomain/41965] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-07 15:54:13,439 DEBUG [Listener at localhost.localdomain/41965] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 15:54:13,457 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:13,457 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:13,457 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:13,459 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:13,459 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:13,459 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-07 15:54:13,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitWithoutPONR 2023-02-07 15:54:13,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:13,463 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:13,463 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:13,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-07 15:54:13,463 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785253463"}]},"ts":"1675785253463"} 2023-02-07 15:54:13,510 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-07 15:54:13,562 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-07 15:54:13,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c9f79cf0473a30691a3c2d31c830aa7, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fadaaa28c739d279eb5c9fd9b2befe27, UNASSIGN}] 2023-02-07 15:54:13,564 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fadaaa28c739d279eb5c9fd9b2befe27, UNASSIGN 2023-02-07 15:54:13,564 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:13,564 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:13,564 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=fadaaa28c739d279eb5c9fd9b2befe27, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:13,565 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785253564"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785253564"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785253564"}]},"ts":"1675785253564"} 2023-02-07 15:54:13,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-07 15:54:13,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure fadaaa28c739d279eb5c9fd9b2befe27, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:13,569 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c9f79cf0473a30691a3c2d31c830aa7, UNASSIGN 2023-02-07 15:54:13,569 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=0c9f79cf0473a30691a3c2d31c830aa7, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:13,569 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785253569"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785253569"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785253569"}]},"ts":"1675785253569"} 2023-02-07 15:54:13,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 0c9f79cf0473a30691a3c2d31c830aa7, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:13,718 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 0c9f79cf0473a30691a3c2d31c830aa7 2023-02-07 15:54:13,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 0c9f79cf0473a30691a3c2d31c830aa7, disabling compactions & flushes 2023-02-07 15:54:13,719 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7. 2023-02-07 15:54:13,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7. 2023-02-07 15:54:13,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7. after waiting 0 ms 2023-02-07 15:54:13,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7. 2023-02-07 15:54:13,731 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999-bottom] to archive 2023-02-07 15:54:13,732 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:13,733 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,735 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,736 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,737 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,738 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,739 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,741 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,743 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29-bottom] to archive 2023-02-07 15:54:13,744 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:13,745 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,746 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,748 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,749 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,750 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,751 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,752 DEBUG [StoreCloser-testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,756 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 15:54:13,756 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:13,756 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7. 2023-02-07 15:54:13,756 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 0c9f79cf0473a30691a3c2d31c830aa7: Waiting for close lock at 1675785253718Running coprocessor pre-close hooks at 1675785253718Disabling compacts and flushes for region at 1675785253718Disabling writes for close at 1675785253719 (+1 ms)Writing region close event to WAL at 1675785253753 (+34 ms)Running coprocessor post-close hooks at 1675785253756 (+3 ms)Closed at 1675785253756 2023-02-07 15:54:13,758 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 0c9f79cf0473a30691a3c2d31c830aa7 2023-02-07 15:54:13,758 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:13,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing fadaaa28c739d279eb5c9fd9b2befe27, disabling compactions & flushes 2023-02-07 15:54:13,758 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27. 2023-02-07 15:54:13,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27. 2023-02-07 15:54:13,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27. after waiting 0 ms 2023-02-07 15:54:13,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27. 2023-02-07 15:54:13,759 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=0c9f79cf0473a30691a3c2d31c830aa7, regionState=CLOSED 2023-02-07 15:54:13,759 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785253758"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785253758"}]},"ts":"1675785253758"} 2023-02-07 15:54:13,760 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999-top] to archive 2023-02-07 15:54:13,760 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:13,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-07 15:54:13,762 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 0c9f79cf0473a30691a3c2d31c830aa7, server=jenkins-hbase12.apache.org,38629,1675785236797 in 189 msec 2023-02-07 15:54:13,764 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c9f79cf0473a30691a3c2d31c830aa7, UNASSIGN in 200 msec 2023-02-07 15:54:13,765 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:13,765 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:13,766 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-07 15:54:13,767 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,768 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,770 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,771 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,773 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29-top] to archive 2023-02-07 15:54:13,774 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:13,775 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,776 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,777 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,779 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,780 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,781 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,782 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:13,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 15:54:13,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:13,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27. 2023-02-07 15:54:13,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for fadaaa28c739d279eb5c9fd9b2befe27: Waiting for close lock at 1675785253758Running coprocessor pre-close hooks at 1675785253758Disabling compacts and flushes for region at 1675785253758Disabling writes for close at 1675785253758Writing region close event to WAL at 1675785253783 (+25 ms)Running coprocessor post-close hooks at 1675785253787 (+4 ms)Closed at 1675785253787 2023-02-07 15:54:13,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:13,789 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=fadaaa28c739d279eb5c9fd9b2befe27, regionState=CLOSED 2023-02-07 15:54:13,789 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785253789"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785253789"}]},"ts":"1675785253789"} 2023-02-07 15:54:13,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-07 15:54:13,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure fadaaa28c739d279eb5c9fd9b2befe27, server=jenkins-hbase12.apache.org,38629,1675785236797 in 224 msec 2023-02-07 15:54:13,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-02-07 15:54:13,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fadaaa28c739d279eb5c9fd9b2befe27, UNASSIGN in 229 msec 2023-02-07 15:54:13,796 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785253796"}]},"ts":"1675785253796"} 2023-02-07 15:54:13,798 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-07 15:54:13,818 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-07 15:54:13,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 359 msec 2023-02-07 15:54:14,066 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:14,067 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:14,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-07 15:54:14,068 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-07 15:54:14,068 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:14,068 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:14,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitWithoutPONR 2023-02-07 15:54:14,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:14,071 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:14,072 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:14,072 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:14,072 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:14,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-07 15:54:14,078 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,078 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7 2023-02-07 15:54:14,078 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:14,080 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/recovered.edits] 2023-02-07 15:54:14,081 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/recovered.edits] 2023-02-07 15:54:14,081 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/recovered.edits] 2023-02-07 15:54:14,083 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/fa5eb71d533d4d66803839f5ca7ff24e to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf1/fa5eb71d533d4d66803839f5ca7ff24e 2023-02-07 15:54:14,084 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/.regioninfo to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/.regioninfo 2023-02-07 15:54:14,084 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/2227cf2356db4b0db8439d6d6653f63a 2023-02-07 15:54:14,085 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/233eb9b69d00434bb007481a745a83c0 2023-02-07 15:54:14,086 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/393a5deb0f90415080fec81cffe4c5e6 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/cf2/393a5deb0f90415080fec81cffe4c5e6 2023-02-07 15:54:14,087 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/33d15303282549ae8de8a01a5cd0899d 2023-02-07 15:54:14,087 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,088 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/6939e3c25b0e4c8b991054375ebf8403 2023-02-07 15:54:14,089 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,089 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/.regioninfo to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/.regioninfo 2023-02-07 15:54:14,090 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/705f4720d12b407f8402e41811956243 2023-02-07 15:54:14,090 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,091 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,091 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/8bb480d936e243fc9e70fe75a2df5999 2023-02-07 15:54:14,091 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/2227cf2356db4b0db8439d6d6653f63a.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,092 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,093 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf1/9df53007e90f462d856925699ebb9c6c 2023-02-07 15:54:14,093 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/233eb9b69d00434bb007481a745a83c0.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,102 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,102 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/33d15303282549ae8de8a01a5cd0899d.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,105 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/6939e3c25b0e4c8b991054375ebf8403.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,105 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,106 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/09deb9b171b64d2685cf8fa5680d26fa 2023-02-07 15:54:14,107 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/705f4720d12b407f8402e41811956243.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,107 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/20ed63de60fc4c39a10df02bb655da27 2023-02-07 15:54:14,109 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/843470a3f5a342dea4ea5b43d8086dab 2023-02-07 15:54:14,109 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/8bb480d936e243fc9e70fe75a2df5999.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,110 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,111 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf1/9df53007e90f462d856925699ebb9c6c.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,111 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/91ed2f39898844728474d4a882035e5d 2023-02-07 15:54:14,112 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,113 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/95069490efc94ba498dab60b38c4f21f 2023-02-07 15:54:14,115 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,115 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/f6845efcabe54979989222bb24ea9b3c 2023-02-07 15:54:14,116 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/09deb9b171b64d2685cf8fa5680d26fa.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,117 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/cf2/fd4e3f8164354994af586cdf89b19e29 2023-02-07 15:54:14,117 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,117 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/20ed63de60fc4c39a10df02bb655da27.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,120 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/843470a3f5a342dea4ea5b43d8086dab.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,120 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,122 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/91ed2f39898844728474d4a882035e5d.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,122 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/recovered.edits/85.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543/recovered.edits/85.seqid 2023-02-07 15:54:14,123 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,123 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,124 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/95069490efc94ba498dab60b38c4f21f.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,125 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/0c9f79cf0473a30691a3c2d31c830aa7/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,126 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/f6845efcabe54979989222bb24ea9b3c.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,127 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/8968ed0b7cff48aa8de66bad06c0dd7f to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf1/8968ed0b7cff48aa8de66bad06c0dd7f 2023-02-07 15:54:14,127 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/fadaaa28c739d279eb5c9fd9b2befe27/cf2/fd4e3f8164354994af586cdf89b19e29.d7d21b347a5059d92fcfbcd4fc307543 2023-02-07 15:54:14,130 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/1b3b017ee69f45d38b9ba9d3ae825891 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/cf2/1b3b017ee69f45d38b9ba9d3ae825891 2023-02-07 15:54:14,130 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/recovered.edits/90.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27/recovered.edits/90.seqid 2023-02-07 15:54:14,131 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:14,133 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/recovered.edits/90.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7/recovered.edits/90.seqid 2023-02-07 15:54:14,134 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitWithoutPONR/0c9f79cf0473a30691a3c2d31c830aa7 2023-02-07 15:54:14,134 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-07 15:54:14,137 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:14,140 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-07 15:54:14,142 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-07 15:54:14,144 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:14,144 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-07 15:54:14,144 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785254144"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:14,144 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785254144"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:14,144 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785254144"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:14,147 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 15:54:14,147 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d7d21b347a5059d92fcfbcd4fc307543, NAME => 'testSplitWithoutPONR,,1675785246242.d7d21b347a5059d92fcfbcd4fc307543.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 0c9f79cf0473a30691a3c2d31c830aa7, NAME => 'testSplitWithoutPONR,,1675785249150.0c9f79cf0473a30691a3c2d31c830aa7.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => fadaaa28c739d279eb5c9fd9b2befe27, NAME => 'testSplitWithoutPONR,41,1675785249150.fadaaa28c739d279eb5c9fd9b2befe27.', STARTKEY => '41', ENDKEY => ''}] 2023-02-07 15:54:14,147 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-07 15:54:14,147 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675785254147"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:14,149 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-07 15:54:14,173 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:14,173 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:14,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-07 15:54:14,240 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:14,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 171 msec 2023-02-07 15:54:14,375 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:14,375 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:14,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-07 15:54:14,376 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-07 15:54:14,389 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=405 (was 395) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:58378 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:58404 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa17a051-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:39216 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:53010 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:58346 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:33826 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:39146 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:39154 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:39126 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:33550 [Waiting for operation #37] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:39192 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:58410 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase12:38629-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:39174 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa17a051-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:39184 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:58416 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:53020 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=864 (was 849) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=370 (was 394), ProcessCount=173 (was 173), AvailableMemoryMB=3178 (was 3316) 2023-02-07 15:54:14,403 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=405, OpenFileDescriptor=864, MaxFileDescriptor=60000, SystemLoadAverage=370, ProcessCount=173, AvailableMemoryMB=3177 2023-02-07 15:54:14,403 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:14,403 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:14,404 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:14,418 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 15:54:14,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 15:54:14,420 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:14,422 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 15:54:14,423 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 15:54:14,425 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:14,425 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc empty. 2023-02-07 15:54:14,426 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:14,426 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-07 15:54:14,443 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-07 15:54:14,444 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => d49006b752d6e5dc24676c9215eaa7dc, NAME => 'testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp 2023-02-07 15:54:14,456 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:14,456 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing d49006b752d6e5dc24676c9215eaa7dc, disabling compactions & flushes 2023-02-07 15:54:14,456 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:14,456 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:14,456 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. after waiting 0 ms 2023-02-07 15:54:14,456 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:14,456 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:14,456 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for d49006b752d6e5dc24676c9215eaa7dc: Waiting for close lock at 1675785254456Disabling compacts and flushes for region at 1675785254456Disabling writes for close at 1675785254456Writing region close event to WAL at 1675785254456Closed at 1675785254456 2023-02-07 15:54:14,459 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 15:54:14,460 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785254459"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785254459"}]},"ts":"1675785254459"} 2023-02-07 15:54:14,461 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 15:54:14,462 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 15:54:14,462 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785254462"}]},"ts":"1675785254462"} 2023-02-07 15:54:14,464 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-07 15:54:14,510 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:54:14,511 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:54:14,511 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:54:14,511 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:54:14,511 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 15:54:14,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN}] 2023-02-07 15:54:14,513 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:14,514 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:54:14,664 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:54:14,665 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=d49006b752d6e5dc24676c9215eaa7dc, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:14,665 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785254665"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785254665"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785254665"}]},"ts":"1675785254665"} 2023-02-07 15:54:14,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:14,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:14,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d49006b752d6e5dc24676c9215eaa7dc, NAME => 'testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:54:14,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:14,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:14,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:14,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:14,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:14,825 INFO [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:14,827 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1 2023-02-07 15:54:14,827 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1 2023-02-07 15:54:14,827 INFO [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d49006b752d6e5dc24676c9215eaa7dc columnFamilyName cf1 2023-02-07 15:54:14,828 INFO [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(310): Store=d49006b752d6e5dc24676c9215eaa7dc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:14,828 INFO [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:14,829 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2 2023-02-07 15:54:14,829 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2 2023-02-07 15:54:14,830 INFO [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d49006b752d6e5dc24676c9215eaa7dc columnFamilyName cf2 2023-02-07 15:54:14,830 INFO [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(310): Store=d49006b752d6e5dc24676c9215eaa7dc/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:14,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:14,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:14,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:14,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:14,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:54:14,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened d49006b752d6e5dc24676c9215eaa7dc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73919547, jitterRate=0.10148708522319794}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:14,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for d49006b752d6e5dc24676c9215eaa7dc: Running coprocessor pre-open hook at 1675785254824Writing region info on filesystem at 1675785254824Initializing all the Stores at 1675785254824Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785254825 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785254825Cleaning up temporary data from old regions at 1675785254832 (+7 ms)Cleaning up detritus from prior splits at 1675785254832Running coprocessor post-open hooks at 1675785254836 (+4 ms)Region opened successfully at 1675785254836 2023-02-07 15:54:14,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc., pid=43, masterSystemTime=1675785254819 2023-02-07 15:54:14,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:14,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:14,839 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=d49006b752d6e5dc24676c9215eaa7dc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:14,839 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785254839"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785254839"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785254839"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785254839"}]},"ts":"1675785254839"} 2023-02-07 15:54:14,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-07 15:54:14,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 in 174 msec 2023-02-07 15:54:14,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-07 15:54:14,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN in 332 msec 2023-02-07 15:54:14,845 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 15:54:14,845 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785254845"}]},"ts":"1675785254845"} 2023-02-07 15:54:14,846 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-07 15:54:14,897 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 15:54:14,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 478 msec 2023-02-07 15:54:14,930 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:14,930 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:14,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:14,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-07 15:54:14,932 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-07 15:54:14,932 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:14,932 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-07 15:54:14,932 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 15:54:14,933 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:14,933 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:14,944 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:14,945 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:14,945 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:14,945 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:14,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:14,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:14,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:14,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:14,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:14,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:14,945 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:14,945 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:14,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:14,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:14,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:14,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:14,946 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:14,946 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:14,947 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:14,947 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:14,947 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:14,947 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:14,947 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:14,947 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:14,947 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:14,947 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:14,947 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:14,947 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:14,947 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:14,947 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:14,947 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:14,947 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. started... 2023-02-07 15:54:14,948 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d49006b752d6e5dc24676c9215eaa7dc 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 15:54:14,959 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:14,959 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:14,959 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:14,959 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:14,959 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:14,959 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:15,360 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/856ecfce29a547c5a19be5b0a7237037 2023-02-07 15:54:15,374 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/94ea16e27c374eca8b62e797e449b5c2 2023-02-07 15:54:15,379 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/856ecfce29a547c5a19be5b0a7237037 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/856ecfce29a547c5a19be5b0a7237037 2023-02-07 15:54:15,385 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/856ecfce29a547c5a19be5b0a7237037, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:54:15,386 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/94ea16e27c374eca8b62e797e449b5c2 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/94ea16e27c374eca8b62e797e449b5c2 2023-02-07 15:54:15,391 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/94ea16e27c374eca8b62e797e449b5c2, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:54:15,392 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d49006b752d6e5dc24676c9215eaa7dc in 444ms, sequenceid=6, compaction requested=false 2023-02-07 15:54:15,392 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-07 15:54:15,393 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d49006b752d6e5dc24676c9215eaa7dc: 2023-02-07 15:54:15,393 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:15,393 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:15,393 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:15,393 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,393 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:15,393 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:15,443 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,443 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:15,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:15,444 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,444 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:15,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:15,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:15,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-07 15:54:15,446 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 15:54:15,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27cab5ef[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:15,446 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,454 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,454 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,454 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,454 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,454 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:15,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,454 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:15,454 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:15,454 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:15,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,455 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:15,455 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:15,455 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:15,455 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:15,455 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:15,455 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:15,464 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:15,464 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,464 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:15,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:15,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:15,465 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:15,475 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:15,475 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:15,475 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:15,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:15,485 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:15,486 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:15,486 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:15,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:15,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:15,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:15,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:15,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:15,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 15:54:15,489 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-07 15:54:15,489 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:15,489 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:15,489 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-07 15:54:15,489 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:15,496 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,496 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,496 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,496 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:15,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:15,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,496 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:15,496 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,496 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:15,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:15,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,496 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:15,497 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:15,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:15,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:15,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:15,497 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:15,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:15,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:15,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:15,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:15,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,499 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,499 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:15,499 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:15,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,513 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,513 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:15,513 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:15,513 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:15,513 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:15,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:15,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-07 15:54:15,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:15,513 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:15,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:15,513 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,513 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,513 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:15,514 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:15,514 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:15,514 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:15,513 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:15,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:15,513 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:15,514 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:15,514 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:15,514 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,514 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,514 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,514 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:15,614 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:15,614 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:15,614 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:15,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:15,635 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:15,635 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:15,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:15,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:15,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-07 15:54:15,638 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-07 15:54:15,638 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:15,638 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-07 15:54:15,638 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 15:54:15,638 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,638 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,650 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:15,650 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:15,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:15,650 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,650 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:15,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:15,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,650 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,650 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:15,651 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:15,651 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:15,651 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:15,651 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:15,651 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:15,651 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:15,652 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:15,651 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:15,652 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:15,652 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:15,652 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:15,652 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:15,652 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:15,652 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:15,652 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:15,652 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:15,652 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:15,652 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:15,652 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. started... 2023-02-07 15:54:15,652 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d49006b752d6e5dc24676c9215eaa7dc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:15,664 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,664 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,665 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,665 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,665 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:15,665 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:15,666 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c 2023-02-07 15:54:15,677 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/bbc9fa2444944eab88470425d3c19a0e 2023-02-07 15:54:15,682 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c 2023-02-07 15:54:15,686 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:54:15,687 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/bbc9fa2444944eab88470425d3c19a0e as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bbc9fa2444944eab88470425d3c19a0e 2023-02-07 15:54:15,692 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bbc9fa2444944eab88470425d3c19a0e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:54:15,693 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d49006b752d6e5dc24676c9215eaa7dc in 41ms, sequenceid=19, compaction requested=false 2023-02-07 15:54:15,693 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d49006b752d6e5dc24676c9215eaa7dc: 2023-02-07 15:54:15,693 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:15,693 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:15,693 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:15,693 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,693 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:15,693 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:15,706 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:15,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:15,707 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,707 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:15,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:15,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:15,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:15,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:15,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-07 15:54:15,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cdeca02[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:15,709 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 15:54:15,709 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,717 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,717 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,717 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,717 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:15,717 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:15,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,718 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:15,717 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,718 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:15,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,718 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:15,718 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:15,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,718 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:15,718 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:15,718 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:15,718 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:15,755 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:15,756 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:15,756 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:15,776 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:15,776 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:15,776 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,776 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:15,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:15,776 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:15,776 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:15,777 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:15,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:15,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:15,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:15,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:15,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:15,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:15,781 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 15:54:15,781 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-07 15:54:15,781 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:15,781 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:15,781 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-07 15:54:15,781 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:15,791 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,791 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,791 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,791 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,791 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:15,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,791 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:15,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,791 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:15,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:15,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,791 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:15,792 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:15,792 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:15,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:15,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,793 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:15,793 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:15,794 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,801 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,801 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:15,801 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,801 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:15,801 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:15,801 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:15,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:15,802 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:15,801 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-07 15:54:15,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:15,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:15,801 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:15,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:15,802 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:15,802 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:15,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:15,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:15,802 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:15,802 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:15,802 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:15,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:15,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:15,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:15,802 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:15,803 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,803 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:15,803 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:15,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:15,803 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,803 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,902 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:15,903 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:15,903 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:15,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:15,919 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:15,919 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:15,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:15,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:15,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-07 15:54:15,921 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-07 15:54:15,922 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:15,922 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-07 15:54:15,922 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 15:54:15,922 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,922 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,934 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,934 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:15,934 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:15,934 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:15,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:15,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:15,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:15,935 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:15,935 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:15,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:15,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:15,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:15,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:15,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:15,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,936 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:15,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:15,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:15,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:15,936 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:15,936 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:15,936 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:15,936 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:15,936 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:15,936 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:15,936 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:15,936 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:15,936 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:15,936 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:15,936 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:15,936 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:15,936 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:15,936 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:15,936 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:15,936 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:15,936 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:15,936 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:15,936 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:15,937 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:15,937 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:15,937 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:15,937 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:15,937 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:15,941 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. started... 2023-02-07 15:54:15,941 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d49006b752d6e5dc24676c9215eaa7dc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:15,948 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,948 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,949 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,949 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:15,949 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:15,949 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:15,954 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 15:54:15,956 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/bf216d7991044332ab73ea8c4a2f6910 2023-02-07 15:54:15,980 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/2216f7ccb2f946af8a8889b812d1e16e 2023-02-07 15:54:15,986 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/bf216d7991044332ab73ea8c4a2f6910 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/bf216d7991044332ab73ea8c4a2f6910 2023-02-07 15:54:15,992 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/bf216d7991044332ab73ea8c4a2f6910, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:54:15,994 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/2216f7ccb2f946af8a8889b812d1e16e as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2216f7ccb2f946af8a8889b812d1e16e 2023-02-07 15:54:15,999 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2216f7ccb2f946af8a8889b812d1e16e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:54:16,000 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d49006b752d6e5dc24676c9215eaa7dc in 59ms, sequenceid=32, compaction requested=true 2023-02-07 15:54:16,000 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d49006b752d6e5dc24676c9215eaa7dc: 2023-02-07 15:54:16,000 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:16,000 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:16,000 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:16,001 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,001 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:16,001 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:16,063 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,063 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:16,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:16,064 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,064 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:16,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:16,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:16,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:16,066 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-07 15:54:16,066 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@760dba95[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:16,066 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 15:54:16,066 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,132 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,132 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,132 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,133 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:16,133 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:16,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,133 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,133 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:16,133 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:16,133 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:16,133 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:16,133 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:16,133 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:16,133 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:16,133 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:16,159 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:16,159 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:16,159 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:16,159 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,159 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:16,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:16,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:16,159 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:16,160 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:16,160 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:16,159 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:16,160 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:16,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:16,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:16,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:16,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 15:54:16,163 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-07 15:54:16,163 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:16,163 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:16,163 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-07 15:54:16,163 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:16,176 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,176 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,176 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:16,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:16,176 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,176 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,176 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,176 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:16,177 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,177 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,177 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:16,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:16,179 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,179 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,179 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,220 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,220 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,220 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,220 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,220 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-07 15:54:16,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:16,220 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:16,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,220 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,220 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:16,220 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:16,220 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,220 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,220 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,220 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,221 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,320 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:16,321 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:16,321 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:16,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:16,337 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:16,337 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:16,338 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:16,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:16,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-07 15:54:16,339 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-07 15:54:16,340 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:16,340 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-07 15:54:16,340 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 15:54:16,340 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,340 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,359 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,359 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,359 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,359 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,360 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,360 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:16,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:16,361 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:16,361 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:16,361 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:16,361 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:16,361 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:16,361 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:16,361 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:16,361 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:16,361 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:16,361 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:16,361 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:16,361 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:16,361 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:16,362 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:16,362 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:16,362 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:16,362 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:16,362 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:16,362 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:16,362 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:16,362 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. started... 2023-02-07 15:54:16,362 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d49006b752d6e5dc24676c9215eaa7dc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:16,369 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,371 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/185e0882083c4299aa67cbd824d76275 2023-02-07 15:54:16,380 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,380 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,380 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:16,380 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,380 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:16,384 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/07a4ac5935e84ad5b322915b2f89fc25 2023-02-07 15:54:16,389 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/185e0882083c4299aa67cbd824d76275 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/185e0882083c4299aa67cbd824d76275 2023-02-07 15:54:16,393 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/185e0882083c4299aa67cbd824d76275, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:54:16,394 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/07a4ac5935e84ad5b322915b2f89fc25 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/07a4ac5935e84ad5b322915b2f89fc25 2023-02-07 15:54:16,398 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/07a4ac5935e84ad5b322915b2f89fc25, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:54:16,399 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d49006b752d6e5dc24676c9215eaa7dc in 37ms, sequenceid=45, compaction requested=true 2023-02-07 15:54:16,399 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d49006b752d6e5dc24676c9215eaa7dc: 2023-02-07 15:54:16,399 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:16,399 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:16,399 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:16,399 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,399 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:16,399 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:16,469 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,469 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:16,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:16,470 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,470 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:16,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:16,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:16,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-07 15:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58a96d4a[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:16,472 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 15:54:16,472 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,502 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,502 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,502 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,503 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,503 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:16,503 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:16,503 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:16,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,503 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:16,503 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:16,503 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:16,503 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:16,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,503 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:16,503 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:16,503 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:16,513 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:16,513 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:16,513 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:16,524 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:16,524 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:16,524 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:16,534 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:16,534 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:16,534 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:16,534 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:16,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:16,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:16,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:16,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:16,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:16,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 15:54:16,538 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-07 15:54:16,539 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:16,539 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:16,539 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-07 15:54:16,539 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:16,548 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,548 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,548 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:16,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:16,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,549 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:16,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,550 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,550 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:16,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,551 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:16,551 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,551 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,559 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,559 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,559 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,559 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:16,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-07 15:54:16,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:16,559 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,559 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,559 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,559 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,559 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,559 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,560 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:16,560 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:16,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,560 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,660 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:16,660 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:16,660 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:16,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:16,676 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:16,676 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:16,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:16,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:16,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-07 15:54:16,678 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-07 15:54:16,678 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:16,678 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-07 15:54:16,679 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 15:54:16,679 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,679 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,685 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,685 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,685 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,685 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,686 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,686 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:16,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,686 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,687 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:16,687 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:16,687 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:16,687 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:16,687 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:16,688 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:16,688 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:16,688 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. started... 2023-02-07 15:54:16,688 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:16,688 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d49006b752d6e5dc24676c9215eaa7dc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:16,696 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,697 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0 2023-02-07 15:54:16,706 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,706 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,706 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:16,707 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,707 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:16,710 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/bfe8678732974167b9f110caf2a5bba9 2023-02-07 15:54:16,714 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0 2023-02-07 15:54:16,718 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:54:16,719 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/bfe8678732974167b9f110caf2a5bba9 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bfe8678732974167b9f110caf2a5bba9 2023-02-07 15:54:16,723 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bfe8678732974167b9f110caf2a5bba9, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:54:16,724 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d49006b752d6e5dc24676c9215eaa7dc in 36ms, sequenceid=58, compaction requested=true 2023-02-07 15:54:16,724 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d49006b752d6e5dc24676c9215eaa7dc: 2023-02-07 15:54:16,724 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:16,724 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:16,724 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:16,724 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,724 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:16,724 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:16,734 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,734 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:16,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:16,735 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:16,735 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:16,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:16,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:16,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-07 15:54:16,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2abede7d[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:16,736 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 15:54:16,737 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,748 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,748 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,748 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:16,749 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,749 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:16,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:16,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:16,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:16,789 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:16,789 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:16,789 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:16,808 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:16,808 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:16,808 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:16,818 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,818 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:16,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:16,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:16,818 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:16,818 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:16,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:16,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:16,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:16,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 15:54:16,821 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-07 15:54:16,821 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:16,821 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:16,821 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-07 15:54:16,821 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:16,833 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,833 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,833 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:16,833 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,833 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,833 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:16,833 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,833 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,834 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:16,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,834 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:16,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:16,836 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,836 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,836 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,850 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:16,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,850 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-07 15:54:16,850 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:16,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:16,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,850 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:16,851 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:16,851 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,851 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:16,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:16,851 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:16,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:16,851 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,851 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,851 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,951 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:16,951 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:16,951 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:16,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:16,967 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:16,967 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:16,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:16,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:16,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-07 15:54:16,970 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-07 15:54:16,970 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:16,970 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-07 15:54:16,970 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 15:54:16,970 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,970 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,980 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:16,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:16,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:16,981 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:16,981 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:16,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:16,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:16,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:16,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,982 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:16,982 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:16,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:16,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:16,982 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 15:54:16,982 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:16,982 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:16,982 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:16,982 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:16,982 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:16,982 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:16,982 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:16,982 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. started... 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:16,982 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:16,983 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d49006b752d6e5dc24676c9215eaa7dc 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:16,991 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,991 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,992 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,992 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:16,992 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:16,992 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:16,992 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/382c6ad5d61d4d64832e29e03c859126 2023-02-07 15:54:17,006 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/cefe27a80c5e463093a7f2e9d63f6577 2023-02-07 15:54:17,011 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/382c6ad5d61d4d64832e29e03c859126 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/382c6ad5d61d4d64832e29e03c859126 2023-02-07 15:54:17,016 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/382c6ad5d61d4d64832e29e03c859126, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:54:17,018 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/cefe27a80c5e463093a7f2e9d63f6577 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/cefe27a80c5e463093a7f2e9d63f6577 2023-02-07 15:54:17,024 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/cefe27a80c5e463093a7f2e9d63f6577, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:54:17,025 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d49006b752d6e5dc24676c9215eaa7dc in 43ms, sequenceid=71, compaction requested=true 2023-02-07 15:54:17,026 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d49006b752d6e5dc24676c9215eaa7dc: 2023-02-07 15:54:17,026 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:17,026 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:17,026 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:17,026 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,026 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 15:54:17,026 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 15:54:17,033 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,033 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:17,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:17,033 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,033 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:17,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:17,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:17,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:17,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:17,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:17,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:17,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-07 15:54:17,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50ca9ff2[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:17,035 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 15:54:17,035 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,043 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,043 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,043 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,044 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,044 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:17,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,044 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:17,044 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:17,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,044 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:17,044 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:17,044 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:17,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,044 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:17,044 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:54:17,044 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 15:54:17,044 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:17,054 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:17,054 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:17,054 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:17,064 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:17,064 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,064 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 15:54:17,065 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:17,065 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:17,064 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:17,065 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 15:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:17,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:17,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:17,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:17,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:17,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:17,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:17,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:17,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:17,067 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 15:54:17,067 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-07 15:54:17,068 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:17,068 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:17,068 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-07 15:54:17,068 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:17,075 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,075 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,075 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:17,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:17,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,075 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:17,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:17,075 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,075 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:17,075 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:17,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:17,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:17,076 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:17,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,076 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,077 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:17,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:17,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:17,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:17,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:17,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 15:54:17,078 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,078 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:17,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:17,079 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:17,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:17,116 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:17,116 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:17,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:17,116 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:17,116 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:17,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:17,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:17,116 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:17,116 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:17,116 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:17,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:17,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:17,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-07 15:54:17,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:17,116 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:17,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:17,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:17,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:17,117 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:17,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:17,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:17,117 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:17,117 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:17,117 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:17,117 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 15:54:17,117 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:17,117 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:17,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:17,118 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,118 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:17,118 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,118 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 15:54:17,118 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 15:54:17,218 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:17,218 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:17,218 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:17,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:17,479 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-07 15:54:17,481 DEBUG [Listener at localhost.localdomain/41965] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:17,482 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,483 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,483 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,483 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,484 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:17,484 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:17,484 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:17,732 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,732 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:17,732 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:17,732 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:17,732 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-07 15:54:17,733 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:17,733 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:17,733 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:17,733 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:17,734 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:17,735 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-07 15:54:17,735 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:17,735 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:17,735 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:17,735 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:17,736 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:17,736 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:17,736 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:17,736 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:17,736 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:17,737 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:17,737 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:17,737 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:17,737 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:17,737 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,738 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:17,738 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,738 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,738 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,738 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 15:54:17,739 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:17,739 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:17,740 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785287740 2023-02-07 15:54:17,741 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:17,741 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,743 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:17,744 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:17,744 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:17,744 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785317744 2023-02-07 15:54:17,744 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785377744 2023-02-07 15:54:17,744 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:54:17,744 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:17,747 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:17,747 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:17,747 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:17,994 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:17,994 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:17,994 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:17,994 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:17,995 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:17,995 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:17,995 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:17,995 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:17,995 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:17,996 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:17,997 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-07 15:54:17,997 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:17,998 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:17,998 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:17,998 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:17,998 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:17,999 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:17,999 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:17,999 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:17,999 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:18,000 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:18,000 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:18,000 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:18,000 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:18,000 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,001 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:18,001 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,001 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,001 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,001 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 15:54:18,001 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:18,002 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:18,005 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785288005 2023-02-07 15:54:18,007 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,007 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:18,009 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:18,009 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:18,009 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:18,009 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785318009 2023-02-07 15:54:18,010 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785378010 2023-02-07 15:54:18,010 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:54:18,010 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:18,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN}] 2023-02-07 15:54:18,012 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:18,012 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:18,013 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:18,260 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,260 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:18,260 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:18,260 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:18,260 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:18,260 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:18,261 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:18,261 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:18,261 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:18,262 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:18,262 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-07 15:54:18,263 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:18,263 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:18,263 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:18,263 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:18,264 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:18,264 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:18,264 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:18,264 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:18,265 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:18,265 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:18,266 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:18,266 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:18,266 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:18,267 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,267 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:18,267 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,267 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,267 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,267 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 15:54:18,268 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:18,268 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:18,269 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785288269 2023-02-07 15:54:18,270 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,270 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:18,272 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:18,272 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:18,272 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:18,273 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785318273 2023-02-07 15:54:18,273 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785378273 2023-02-07 15:54:18,273 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:54:18,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN}] 2023-02-07 15:54:18,273 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:18,276 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:18,276 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:18,277 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785258276"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785258276"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785258276"}]},"ts":"1675785258276"} 2023-02-07 15:54:18,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:18,278 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:18,278 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:18,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:18,278 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:18,523 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,523 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:18,523 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:18,523 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:18,524 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:18,525 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:18,525 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:18,525 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:18,525 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:18,526 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:18,527 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 15:54:18,527 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-07 15:54:18,527 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:18,527 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:18,527 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:18,528 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:18,528 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:18,528 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:18,528 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:18,528 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:18,529 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:18,529 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:18,529 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:18,529 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:18,530 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:18,531 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,531 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:18,531 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:18,531 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,531 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,531 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,531 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:18,531 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:18,532 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 15:54:18,532 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:18,533 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:18,533 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc to restore RIT 2023-02-07 15:54:18,533 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785288533 2023-02-07 15:54:18,534 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:18,534 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:18,536 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:18,536 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSING, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:18,537 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:18,537 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785318537 2023-02-07 15:54:18,537 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785378537 2023-02-07 15:54:18,537 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:54:18,537 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:18,537 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785258537"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785258537"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785258537"}]},"ts":"1675785258537"} 2023-02-07 15:54:18,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:18,539 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:18,693 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:18,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing d49006b752d6e5dc24676c9215eaa7dc, disabling compactions & flushes 2023-02-07 15:54:18,693 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:18,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:18,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. after waiting 0 ms 2023-02-07 15:54:18,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:18,694 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing d49006b752d6e5dc24676c9215eaa7dc 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 15:54:18,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/c8bf3e747de84c8893387371c5bb6ad4 2023-02-07 15:54:18,716 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/10f85619083043bfb02353d3a7742e33 2023-02-07 15:54:18,720 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/c8bf3e747de84c8893387371c5bb6ad4 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/c8bf3e747de84c8893387371c5bb6ad4 2023-02-07 15:54:18,725 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/c8bf3e747de84c8893387371c5bb6ad4, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:54:18,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/10f85619083043bfb02353d3a7742e33 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/10f85619083043bfb02353d3a7742e33 2023-02-07 15:54:18,730 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/10f85619083043bfb02353d3a7742e33, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:54:18,731 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d49006b752d6e5dc24676c9215eaa7dc in 38ms, sequenceid=82, compaction requested=true 2023-02-07 15:54:18,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 15:54:18,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:18,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:18,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for d49006b752d6e5dc24676c9215eaa7dc: Waiting for close lock at 1675785258693Running coprocessor pre-close hooks at 1675785258693Disabling compacts and flushes for region at 1675785258693Disabling writes for close at 1675785258693Obtaining lock to block concurrent updates at 1675785258694 (+1 ms)Preparing flush snapshotting stores in d49006b752d6e5dc24676c9215eaa7dc at 1675785258694Finished memstore snapshotting testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675785258694Flushing stores of testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. at 1675785258694Flushing d49006b752d6e5dc24676c9215eaa7dc/cf1: creating writer at 1675785258695 (+1 ms)Flushing d49006b752d6e5dc24676c9215eaa7dc/cf1: appending metadata at 1675785258698 (+3 ms)Flushing d49006b752d6e5dc24676c9215eaa7dc/cf1: closing flushed file at 1675785258698Flushing d49006b752d6e5dc24676c9215eaa7dc/cf2: creating writer at 1675785258708 (+10 ms)Flushing d49006b752d6e5dc24676c9215eaa7dc/cf2: appending metadata at 1675785258710 (+2 ms)Flushing d49006b752d6e5dc24676c9215eaa7dc/cf2: closing flushed file at 1675785258710Flushing d49006b752d6e5dc24676c9215eaa7dc/cf1: reopening flushed file at 1675785258721 (+11 ms)Flushing d49006b752d6e5dc24676c9215eaa7dc/cf2: reopening flushed file at 1675785258726 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d49006b752d6e5dc24676c9215eaa7dc in 38ms, sequenceid=82, compaction requested=true at 1675785258731 (+5 ms)Writing region close event to WAL at 1675785258732 (+1 ms)Running coprocessor post-close hooks at 1675785258736 (+4 ms)Closed at 1675785258736 2023-02-07 15:54:18,739 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:18,740 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED 2023-02-07 15:54:18,740 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785258740"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785258740"}]},"ts":"1675785258740"} 2023-02-07 15:54:18,741 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:18,741 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:18,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:18,741 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:18,787 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,787 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:18,787 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:18,787 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:18,788 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:18,788 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:18,788 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:18,788 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:18,788 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:18,789 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:18,790 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 15:54:18,790 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-07 15:54:18,790 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-07 15:54:18,790 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:18,790 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:18,790 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:18,790 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:18,791 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:18,791 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:18,791 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:18,791 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:18,791 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:18,792 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:18,792 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:18,792 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:18,792 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:18,793 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,793 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:18,793 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:18,793 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:18,793 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,793 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,793 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:18,793 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:18,793 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:18,793 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:18,793 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 15:54:18,794 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:18,795 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:18,795 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc to restore RIT 2023-02-07 15:54:18,795 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785288795 2023-02-07 15:54:18,796 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:18,796 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:18,798 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:18,799 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:18,799 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:18,799 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785318799 2023-02-07 15:54:18,799 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785378799 2023-02-07 15:54:18,799 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:54:18,799 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:18,799 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785258799"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785258799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785258799"}]},"ts":"1675785258799"} 2023-02-07 15:54:18,801 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:18,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-07 15:54:18,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 in 262 msec 2023-02-07 15:54:18,804 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:18,804 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:18,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:18,804 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:19,049 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,049 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:19,049 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:19,049 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:19,049 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:19,050 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:19,050 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:19,050 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:19,050 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:19,051 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:19,051 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 15:54:19,051 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-07 15:54:19,052 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:19,052 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:19,053 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:19,053 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:19,053 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:19,053 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:19,054 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:19,054 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:19,054 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:19,055 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:19,055 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:19,055 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:19,055 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:19,055 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:19,056 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,057 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:19,057 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:19,057 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:19,057 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,057 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,057 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,057 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:19,057 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:19,057 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 15:54:19,058 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:19,059 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:19,059 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc to restore RIT 2023-02-07 15:54:19,060 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785289060 2023-02-07 15:54:19,061 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:19,061 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:19,063 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:19,063 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:19,064 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:19,064 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785319064 2023-02-07 15:54:19,064 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785379064 2023-02-07 15:54:19,064 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:54:19,064 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:19,066 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:19,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-07 15:54:19,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN in 793 msec 2023-02-07 15:54:19,072 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:19,072 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:19,072 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:19,314 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,314 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:19,314 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:19,314 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:19,317 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:19,317 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:19,317 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:19,317 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:19,317 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:19,318 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:19,319 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 15:54:19,319 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:54:19,319 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:19,320 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:19,320 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:19,320 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:19,320 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:19,320 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:19,321 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:19,321 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:19,321 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:19,321 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:19,321 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:19,321 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:19,322 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:19,322 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:19,322 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,322 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:19,322 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:19,322 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:19,323 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:19,323 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:19,323 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 15:54:19,323 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:19,324 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:19,324 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785289324 2023-02-07 15:54:19,326 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,326 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:19,328 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:19,328 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:19,328 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:19,328 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785319328 2023-02-07 15:54:19,329 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785379329 2023-02-07 15:54:19,329 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:54:19,329 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,329 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,329 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,334 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:19,343 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=d49006b752d6e5dc24676c9215eaa7dc, threads=14 2023-02-07 15:54:19,344 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/07a4ac5935e84ad5b322915b2f89fc25 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,344 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/10f85619083043bfb02353d3a7742e33 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,344 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2216f7ccb2f946af8a8889b812d1e16e for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,344 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/94ea16e27c374eca8b62e797e449b5c2 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,345 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bbc9fa2444944eab88470425d3c19a0e for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,345 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bfe8678732974167b9f110caf2a5bba9 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,345 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/cefe27a80c5e463093a7f2e9d63f6577 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,345 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,345 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/185e0882083c4299aa67cbd824d76275 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,345 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/382c6ad5d61d4d64832e29e03c859126 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,345 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,346 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/bf216d7991044332ab73ea8c4a2f6910 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,346 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/c8bf3e747de84c8893387371c5bb6ad4 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,346 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/856ecfce29a547c5a19be5b0a7237037 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,508 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2216f7ccb2f946af8a8889b812d1e16e for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,541 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bfe8678732974167b9f110caf2a5bba9 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,545 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,556 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,574 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/856ecfce29a547c5a19be5b0a7237037 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,575 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/cefe27a80c5e463093a7f2e9d63f6577 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,577 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/94ea16e27c374eca8b62e797e449b5c2 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,585 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/185e0882083c4299aa67cbd824d76275 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,589 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/382c6ad5d61d4d64832e29e03c859126 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,589 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/07a4ac5935e84ad5b322915b2f89fc25 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,589 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/bf216d7991044332ab73ea8c4a2f6910 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,593 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/c8bf3e747de84c8893387371c5bb6ad4 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,597 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bbc9fa2444944eab88470425d3c19a0e for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,891 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/10f85619083043bfb02353d3a7742e33 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:19,891 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region d49006b752d6e5dc24676c9215eaa7dc Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 15:54:19,928 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:19,929 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:19,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:19,929 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:20,079 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,079 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:20,079 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:20,079 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:20,080 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:20,080 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:20,080 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:20,080 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:20,080 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:20,081 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:20,082 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-07 15:54:20,082 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:54:20,082 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:20,082 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:20,083 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:20,083 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:20,083 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:20,083 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:20,084 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:20,084 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:20,084 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:20,084 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:20,084 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:20,085 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:20,085 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:20,085 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:20,085 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,085 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:20,086 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:20,086 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:20,086 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,086 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,086 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,086 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:20,086 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:20,086 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 15:54:20,087 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:20,087 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:20,088 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785290088 2023-02-07 15:54:20,089 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,089 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:20,092 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:20,092 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:20,092 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:20,092 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785320092 2023-02-07 15:54:20,092 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785380092 2023-02-07 15:54:20,093 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:20,102 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=d49006b752d6e5dc24676c9215eaa7dc, threads=14 2023-02-07 15:54:20,103 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/07a4ac5935e84ad5b322915b2f89fc25 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,103 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/10f85619083043bfb02353d3a7742e33 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,104 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2216f7ccb2f946af8a8889b812d1e16e for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,107 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/94ea16e27c374eca8b62e797e449b5c2 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,109 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bbc9fa2444944eab88470425d3c19a0e for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,109 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bfe8678732974167b9f110caf2a5bba9 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,109 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,109 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/cefe27a80c5e463093a7f2e9d63f6577 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,112 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/185e0882083c4299aa67cbd824d76275 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,113 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/382c6ad5d61d4d64832e29e03c859126 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,113 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,114 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/856ecfce29a547c5a19be5b0a7237037 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,114 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/bf216d7991044332ab73ea8c4a2f6910 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,115 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/c8bf3e747de84c8893387371c5bb6ad4 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,232 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2216f7ccb2f946af8a8889b812d1e16e for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,241 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/cefe27a80c5e463093a7f2e9d63f6577 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,253 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/856ecfce29a547c5a19be5b0a7237037 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,262 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/94ea16e27c374eca8b62e797e449b5c2 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,264 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/382c6ad5d61d4d64832e29e03c859126 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,285 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/07a4ac5935e84ad5b322915b2f89fc25 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,297 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/c8bf3e747de84c8893387371c5bb6ad4 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,301 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/10f85619083043bfb02353d3a7742e33 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,315 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,316 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/bf216d7991044332ab73ea8c4a2f6910 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,317 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bbc9fa2444944eab88470425d3c19a0e for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,621 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/185e0882083c4299aa67cbd824d76275 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,621 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,672 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bfe8678732974167b9f110caf2a5bba9 for region: d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:20,672 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region d49006b752d6e5dc24676c9215eaa7dc Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 15:54:20,711 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:20,717 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/b992767259acdadc811d9727ba201a05/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 15:54:20,719 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/c1f9597ce00dcf3521f395fe441cc65c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 15:54:20,719 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:20,719 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:20,719 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:20,843 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,843 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:20,843 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:20,843 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:20,844 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:20,844 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:20,844 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:20,844 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:20,844 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:20,845 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:20,846 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-07 15:54:20,846 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:54:20,846 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:20,846 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:20,846 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:20,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:20,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:20,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:20,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:20,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:20,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:20,848 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:20,848 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:20,848 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:20,848 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:20,848 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:20,849 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,849 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:20,849 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:20,851 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:20,851 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,851 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,851 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,851 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:20,851 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:20,851 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 15:54:20,852 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:20,852 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:20,853 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785290853 2023-02-07 15:54:20,854 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,854 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:20,856 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:20,856 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:20,856 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:20,857 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785320857 2023-02-07 15:54:20,857 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785380857 2023-02-07 15:54:20,857 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:54:20,859 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:20,863 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:20,863 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:20,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:20,863 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:21,108 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,108 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:21,108 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:21,108 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:21,108 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:21,108 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:21,108 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:21,109 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:21,109 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:21,110 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:21,111 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-07 15:54:21,111 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:54:21,111 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:21,111 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:21,112 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:21,112 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:21,112 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:21,113 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:21,113 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:21,113 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:21,114 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:21,114 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:21,114 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:21,115 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:21,115 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:21,115 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:21,116 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,116 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:21,116 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:21,116 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:21,116 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,116 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,116 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,117 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:21,117 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:21,117 DEBUG [Listener at localhost.localdomain/41965] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,117 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-07 15:54:21,118 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:21,118 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:21,119 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785291119 2023-02-07 15:54:21,121 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,121 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:21,123 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:21,124 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:21,124 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:21,124 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785321124 2023-02-07 15:54:21,124 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785381124 2023-02-07 15:54:21,124 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:21,125 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:21,136 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:21,136 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 15:54:21,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:21,136 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:21,375 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,376 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:21,376 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:21,376 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:21,376 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:21,376 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:21,376 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:21,376 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:21,376 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:21,377 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 15:54:21,378 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-07 15:54:21,378 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:54:21,378 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:21,379 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:21,379 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:21,379 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:21,379 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:21,380 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:21,380 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:21,380 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:21,380 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:21,380 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:21,381 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:21,381 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:21,381 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:21,382 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:21,382 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,382 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:21,382 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:21,382 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:21,382 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,383 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,383 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,383 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:21,383 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:21,383 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 15:54:21,384 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:21,384 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:21,384 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785291384 2023-02-07 15:54:21,387 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:21,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,389 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:21,390 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:21,390 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:21,390 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785321390 2023-02-07 15:54:21,390 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785381390 2023-02-07 15:54:21,390 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:54:21,390 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:21,393 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:21,393 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 15:54:21,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:21,393 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:21,641 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,641 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:21,641 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:21,641 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:21,642 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:21,642 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:21,642 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:21,642 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:21,642 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:21,643 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:21,643 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-07 15:54:21,644 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:54:21,644 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:21,644 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:21,644 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:21,644 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:21,644 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:21,645 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:21,645 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:21,645 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:21,645 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:21,645 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:21,645 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:21,646 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:21,646 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:21,646 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:21,646 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,647 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:21,647 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:21,647 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:21,647 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,647 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,647 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,647 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:21,647 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:21,647 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 15:54:21,648 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:21,648 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:21,649 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785291649 2023-02-07 15:54:21,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,651 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:21,653 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:21,653 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:21,654 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:21,654 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785321654 2023-02-07 15:54:21,654 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785381654 2023-02-07 15:54:21,654 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:21,654 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:21,657 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:21,657 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 15:54:21,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:21,657 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:21,904 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,904 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:21,904 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:21,905 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:21,905 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:21,905 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:21,905 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:21,905 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:21,905 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:21,906 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:21,907 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-07 15:54:21,907 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:54:21,907 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:21,908 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:21,908 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:21,908 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:21,909 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:21,909 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:21,909 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:21,909 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:21,909 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:21,909 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:21,910 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:21,910 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:21,910 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:21,910 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:21,910 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,910 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:21,911 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:21,911 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:21,911 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,911 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,911 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,911 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:21,911 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:21,911 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 15:54:21,912 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:21,912 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:21,913 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785291913 2023-02-07 15:54:21,914 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:21,914 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:21,916 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:21,917 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:21,917 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:21,917 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785321917 2023-02-07 15:54:21,917 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785381917 2023-02-07 15:54:21,917 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-07 15:54:21,917 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:21,921 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN] 2023-02-07 15:54:21,921 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:21,921 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 15:54:21,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:21,921 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:22,167 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,167 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:22,167 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:22,167 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:22,167 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:22,168 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:22,168 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:22,168 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:22,168 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:22,168 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:22,169 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-07 15:54:22,169 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:54:22,169 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:54:22,169 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-07 15:54:22,169 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:22,170 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:22,170 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:22,170 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:22,170 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:22,170 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:22,170 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:22,170 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:22,171 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:22,171 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:22,171 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:22,171 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:22,171 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:22,172 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,172 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:22,172 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:22,172 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:22,172 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:22,173 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,173 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,173 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,173 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:22,173 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:22,173 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:22,173 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 15:54:22,174 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:22,174 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:22,175 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc to restore RIT 2023-02-07 15:54:22,175 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785292175 2023-02-07 15:54:22,176 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:22,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,177 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:22,179 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:22,180 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:22,180 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:22,180 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785322180 2023-02-07 15:54:22,180 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785382180 2023-02-07 15:54:22,180 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:22,180 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:22,185 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:22,185 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 15:54:22,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:22,185 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:22,430 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,430 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:22,430 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:22,430 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:22,430 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:22,431 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:22,431 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:22,431 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:22,431 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:22,432 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:22,432 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 15:54:22,432 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-07 15:54:22,432 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:22,433 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:22,433 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:22,433 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:22,433 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:22,433 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:22,434 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:22,434 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:22,434 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:22,434 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:22,434 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:22,434 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:22,435 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:22,435 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,435 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:22,435 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:22,435 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,435 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,435 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,435 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:22,435 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-07 15:54:22,436 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:22,437 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:22,437 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc to restore RIT 2023-02-07 15:54:22,441 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785292441 2023-02-07 15:54:22,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,442 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:22,443 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:22,446 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:22,446 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:22,446 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:22,446 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785322446 2023-02-07 15:54:22,446 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785382446 2023-02-07 15:54:22,446 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:22,447 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:22,450 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:22,450 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 15:54:22,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:22,450 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:22,697 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,697 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:22,697 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:22,697 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:22,698 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:22,698 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:22,698 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:22,698 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:22,698 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:22,699 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:22,700 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-07 15:54:22,700 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-07 15:54:22,700 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:22,700 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:22,700 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:22,701 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:22,701 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:22,701 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:22,701 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:22,701 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:22,702 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:22,702 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:22,702 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:22,702 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:22,702 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:22,702 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,702 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:22,703 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:22,703 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,703 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,703 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,703 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:22,703 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-07 15:54:22,704 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:22,704 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:22,704 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc to restore RIT 2023-02-07 15:54:22,704 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785292704 2023-02-07 15:54:22,706 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:22,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,707 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:22,709 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:22,709 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:22,709 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:22,710 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785322709 2023-02-07 15:54:22,710 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785382710 2023-02-07 15:54:22,710 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:54:22,710 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:22,712 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:22,712 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 15:54:22,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:22,712 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:22,960 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,960 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:22,960 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:22,960 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:22,961 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:22,961 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:22,961 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:22,961 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:22,961 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:22,962 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:22,963 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-07 15:54:22,963 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-07 15:54:22,963 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:22,963 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:22,964 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:22,964 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:22,964 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:22,964 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:22,964 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:22,964 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:22,965 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:22,965 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:22,965 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:22,965 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:22,965 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:22,966 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,966 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:22,966 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:22,966 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,966 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,966 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,966 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:54:22,966 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-07 15:54:22,967 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:22,967 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:22,967 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc to restore RIT 2023-02-07 15:54:22,969 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785292969 2023-02-07 15:54:22,970 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:22,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:22,972 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:22,974 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:22,975 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:22,975 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:22,975 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785322975 2023-02-07 15:54:22,975 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785382975 2023-02-07 15:54:22,975 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-07 15:54:22,975 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:54:22,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c exec-time=5.4940 sec 2023-02-07 15:54:22,980 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:22,981 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN; state=CLOSED, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=true 2023-02-07 15:54:23,131 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:54:23,133 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=d49006b752d6e5dc24676c9215eaa7dc, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:23,133 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785263133"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785263133"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785263133"}]},"ts":"1675785263133"} 2023-02-07 15:54:23,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:23,135 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:54:23,135 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:23,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:54:23,135 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:54:23,225 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:23,225 WARN [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-07 15:54:23,225 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:54:23,226 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:54:23,226 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:54:23,226 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:54:23,226 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:54:23,226 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:54:23,226 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:54:23,226 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:54:23,227 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:54:23,228 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-07 15:54:23,228 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:54:23,228 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:54:23,229 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:54:23,229 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:54:23,229 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:23,229 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:54:23,229 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:23,229 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:54:23,230 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:23,230 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:54:23,230 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:23,230 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:54:23,230 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:23,230 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:54:23,231 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:23,231 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:54:23,231 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:23,231 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-07 15:54:23,232 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:54:23,232 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:54:23,232 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc to restore RIT 2023-02-07 15:54:23,232 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785293232 2023-02-07 15:54:23,233 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:54:23,233 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:54:23,236 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:23,236 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=d49006b752d6e5dc24676c9215eaa7dc, regionState=OPENING, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:54:23,236 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:54:23,236 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785323236 2023-02-07 15:54:23,236 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785383236 2023-02-07 15:54:23,236 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:54:23,236 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=d49006b752d6e5dc24676c9215eaa7dc, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:23,237 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785263236"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785263236"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785263236"}]},"ts":"1675785263236"} 2023-02-07 15:54:23,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:23,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:23,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d49006b752d6e5dc24676c9215eaa7dc, NAME => 'testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:54:23,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:23,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:23,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:23,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:23,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:23,396 INFO [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:23,397 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1 2023-02-07 15:54:23,397 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1 2023-02-07 15:54:23,398 INFO [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d49006b752d6e5dc24676c9215eaa7dc columnFamilyName cf1 2023-02-07 15:54:23,404 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c 2023-02-07 15:54:23,408 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/185e0882083c4299aa67cbd824d76275 2023-02-07 15:54:23,411 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/382c6ad5d61d4d64832e29e03c859126 2023-02-07 15:54:23,415 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0 2023-02-07 15:54:23,419 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/856ecfce29a547c5a19be5b0a7237037 2023-02-07 15:54:23,424 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/bf216d7991044332ab73ea8c4a2f6910 2023-02-07 15:54:23,429 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/c8bf3e747de84c8893387371c5bb6ad4 2023-02-07 15:54:23,429 INFO [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(310): Store=d49006b752d6e5dc24676c9215eaa7dc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:23,429 INFO [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:23,430 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2 2023-02-07 15:54:23,430 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2 2023-02-07 15:54:23,431 INFO [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d49006b752d6e5dc24676c9215eaa7dc columnFamilyName cf2 2023-02-07 15:54:23,439 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/07a4ac5935e84ad5b322915b2f89fc25 2023-02-07 15:54:23,444 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/10f85619083043bfb02353d3a7742e33 2023-02-07 15:54:23,448 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2216f7ccb2f946af8a8889b812d1e16e 2023-02-07 15:54:23,452 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/94ea16e27c374eca8b62e797e449b5c2 2023-02-07 15:54:23,456 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bbc9fa2444944eab88470425d3c19a0e 2023-02-07 15:54:23,460 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bfe8678732974167b9f110caf2a5bba9 2023-02-07 15:54:23,465 DEBUG [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/cefe27a80c5e463093a7f2e9d63f6577 2023-02-07 15:54:23,465 INFO [StoreOpener-d49006b752d6e5dc24676c9215eaa7dc-1] regionserver.HStore(310): Store=d49006b752d6e5dc24676c9215eaa7dc/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:23,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:23,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:23,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.splits 2023-02-07 15:54:23,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:23,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:23,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened d49006b752d6e5dc24676c9215eaa7dc; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68976626, jitterRate=0.027831822633743286}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:23,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for d49006b752d6e5dc24676c9215eaa7dc: Running coprocessor pre-open hook at 1675785263395Writing region info on filesystem at 1675785263395Initializing all the Stores at 1675785263396 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785263396Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785263396Cleaning up temporary data from old regions at 1675785263467 (+71 ms)Cleaning up detritus from prior splits at 1675785263468 (+1 ms)Running coprocessor post-open hooks at 1675785263471 (+3 ms)Region opened successfully at 1675785263471 2023-02-07 15:54:23,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc., pid=48, masterSystemTime=1675785263391 2023-02-07 15:54:23,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:23,473 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:54:23,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:54:23,473 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:54:23,474 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-07 15:54:23,474 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-07 15:54:23,474 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1912): d49006b752d6e5dc24676c9215eaa7dc/cf1 is initiating minor compaction (all files) 2023-02-07 15:54:23,474 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1912): d49006b752d6e5dc24676c9215eaa7dc/cf2 is initiating minor compaction (all files) 2023-02-07 15:54:23,474 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of d49006b752d6e5dc24676c9215eaa7dc/cf1 in testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:23,474 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2258): Starting compaction of d49006b752d6e5dc24676c9215eaa7dc/cf2 in testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:23,474 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/94ea16e27c374eca8b62e797e449b5c2, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bbc9fa2444944eab88470425d3c19a0e, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2216f7ccb2f946af8a8889b812d1e16e, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/07a4ac5935e84ad5b322915b2f89fc25, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bfe8678732974167b9f110caf2a5bba9, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/cefe27a80c5e463093a7f2e9d63f6577, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/10f85619083043bfb02353d3a7742e33] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp, totalSize=34.9 K 2023-02-07 15:54:23,474 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/856ecfce29a547c5a19be5b0a7237037, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/bf216d7991044332ab73ea8c4a2f6910, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/185e0882083c4299aa67cbd824d76275, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/382c6ad5d61d4d64832e29e03c859126, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/c8bf3e747de84c8893387371c5bb6ad4] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp, totalSize=34.9 K 2023-02-07 15:54:23,475 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 856ecfce29a547c5a19be5b0a7237037, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785254923 2023-02-07 15:54:23,475 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 94ea16e27c374eca8b62e797e449b5c2, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785254923 2023-02-07 15:54:23,475 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 0e5b64bb0cf041aa8dcc079d6b8a108c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785255616 2023-02-07 15:54:23,475 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting bbc9fa2444944eab88470425d3c19a0e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785255616 2023-02-07 15:54:23,475 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting bf216d7991044332ab73ea8c4a2f6910, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785255905 2023-02-07 15:54:23,475 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 2216f7ccb2f946af8a8889b812d1e16e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785255905 2023-02-07 15:54:23,475 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 185e0882083c4299aa67cbd824d76275, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785256322 2023-02-07 15:54:23,475 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 07a4ac5935e84ad5b322915b2f89fc25, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785256322 2023-02-07 15:54:23,476 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 594a8a32ac1d49dfb0523a4cad9e3fb0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785256662 2023-02-07 15:54:23,476 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting bfe8678732974167b9f110caf2a5bba9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785256662 2023-02-07 15:54:23,476 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 382c6ad5d61d4d64832e29e03c859126, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785256953 2023-02-07 15:54:23,476 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting cefe27a80c5e463093a7f2e9d63f6577, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785256953 2023-02-07 15:54:23,476 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting c8bf3e747de84c8893387371c5bb6ad4, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675785257220 2023-02-07 15:54:23,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:23,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:23,477 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 10f85619083043bfb02353d3a7742e33, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675785257220 2023-02-07 15:54:23,479 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=d49006b752d6e5dc24676c9215eaa7dc, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:23,479 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785263479"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785263479"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785263479"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785263479"}]},"ts":"1675785263479"} 2023-02-07 15:54:23,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-07 15:54:23,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 in 242 msec 2023-02-07 15:54:23,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN in 1.5620 sec 2023-02-07 15:54:23,487 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-07 15:54:23,490 DEBUG [Listener at localhost.localdomain/41965] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-07 15:54:23,491 INFO [Listener at localhost.localdomain/41965] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-07 15:54:23,493 INFO [Listener at localhost.localdomain/41965] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-07 15:54:23,494 INFO [Listener at localhost.localdomain/41965] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-07 15:54:23,494 INFO [Listener at localhost.localdomain/41965] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-07 15:54:23,515 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] throttle.PressureAwareThroughputController(145): d49006b752d6e5dc24676c9215eaa7dc#cf2#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:23,515 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] throttle.PressureAwareThroughputController(145): d49006b752d6e5dc24676c9215eaa7dc#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:23,539 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf1/0af5c2ee62fe4e1f9e0fae6f29d94624 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0af5c2ee62fe4e1f9e0fae6f29d94624 2023-02-07 15:54:23,540 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/.tmp/cf2/2fcba979b8814dd2a7eec279bc8c8238 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2fcba979b8814dd2a7eec279bc8c8238 2023-02-07 15:54:23,550 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-07 15:54:23,550 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d49006b752d6e5dc24676c9215eaa7dc/cf2 of d49006b752d6e5dc24676c9215eaa7dc into 2fcba979b8814dd2a7eec279bc8c8238(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:23,550 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for d49006b752d6e5dc24676c9215eaa7dc: 2023-02-07 15:54:23,550 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc., storeName=d49006b752d6e5dc24676c9215eaa7dc/cf2, priority=9, startTime=1675785263473; duration=0sec 2023-02-07 15:54:23,555 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d49006b752d6e5dc24676c9215eaa7dc/cf1 of d49006b752d6e5dc24676c9215eaa7dc into 0af5c2ee62fe4e1f9e0fae6f29d94624(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:23,555 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:23,555 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for d49006b752d6e5dc24676c9215eaa7dc: 2023-02-07 15:54:23,555 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc., storeName=d49006b752d6e5dc24676c9215eaa7dc/cf1, priority=9, startTime=1675785263472; duration=0sec 2023-02-07 15:54:23,555 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:23,563 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:23,563 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:23,563 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:23,564 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:23,565 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:23,565 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-07 15:54:23,565 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testRollbackAndDoubleExecution 2023-02-07 15:54:23,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:23,568 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:23,568 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:23,569 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785263569"}]},"ts":"1675785263569"} 2023-02-07 15:54:23,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-07 15:54:23,570 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-07 15:54:23,628 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-07 15:54:23,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN}] 2023-02-07 15:54:23,630 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN 2023-02-07 15:54:23,631 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:23,631 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785263631"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785263631"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785263631"}]},"ts":"1675785263631"} 2023-02-07 15:54:23,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:23,669 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:23,669 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:23,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-07 15:54:23,784 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:23,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing d49006b752d6e5dc24676c9215eaa7dc, disabling compactions & flushes 2023-02-07 15:54:23,784 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:23,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:23,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. after waiting 0 ms 2023-02-07 15:54:23,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:23,785 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/856ecfce29a547c5a19be5b0a7237037, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/bf216d7991044332ab73ea8c4a2f6910, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/185e0882083c4299aa67cbd824d76275, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/382c6ad5d61d4d64832e29e03c859126, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/c8bf3e747de84c8893387371c5bb6ad4] to archive 2023-02-07 15:54:23,786 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:23,787 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/856ecfce29a547c5a19be5b0a7237037 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/856ecfce29a547c5a19be5b0a7237037 2023-02-07 15:54:23,788 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0e5b64bb0cf041aa8dcc079d6b8a108c 2023-02-07 15:54:23,790 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/bf216d7991044332ab73ea8c4a2f6910 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/bf216d7991044332ab73ea8c4a2f6910 2023-02-07 15:54:23,791 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/185e0882083c4299aa67cbd824d76275 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/185e0882083c4299aa67cbd824d76275 2023-02-07 15:54:23,793 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/594a8a32ac1d49dfb0523a4cad9e3fb0 2023-02-07 15:54:23,794 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/382c6ad5d61d4d64832e29e03c859126 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/382c6ad5d61d4d64832e29e03c859126 2023-02-07 15:54:23,795 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/c8bf3e747de84c8893387371c5bb6ad4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/c8bf3e747de84c8893387371c5bb6ad4 2023-02-07 15:54:23,800 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/94ea16e27c374eca8b62e797e449b5c2, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bbc9fa2444944eab88470425d3c19a0e, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2216f7ccb2f946af8a8889b812d1e16e, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/07a4ac5935e84ad5b322915b2f89fc25, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bfe8678732974167b9f110caf2a5bba9, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/cefe27a80c5e463093a7f2e9d63f6577, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/10f85619083043bfb02353d3a7742e33] to archive 2023-02-07 15:54:23,800 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:23,802 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/94ea16e27c374eca8b62e797e449b5c2 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/94ea16e27c374eca8b62e797e449b5c2 2023-02-07 15:54:23,803 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bbc9fa2444944eab88470425d3c19a0e to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bbc9fa2444944eab88470425d3c19a0e 2023-02-07 15:54:23,804 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2216f7ccb2f946af8a8889b812d1e16e to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2216f7ccb2f946af8a8889b812d1e16e 2023-02-07 15:54:23,805 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/07a4ac5935e84ad5b322915b2f89fc25 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/07a4ac5935e84ad5b322915b2f89fc25 2023-02-07 15:54:23,806 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bfe8678732974167b9f110caf2a5bba9 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/bfe8678732974167b9f110caf2a5bba9 2023-02-07 15:54:23,807 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/cefe27a80c5e463093a7f2e9d63f6577 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/cefe27a80c5e463093a7f2e9d63f6577 2023-02-07 15:54:23,808 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/10f85619083043bfb02353d3a7742e33 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/10f85619083043bfb02353d3a7742e33 2023-02-07 15:54:23,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 15:54:23,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:23,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc. 2023-02-07 15:54:23,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for d49006b752d6e5dc24676c9215eaa7dc: Waiting for close lock at 1675785263784Running coprocessor pre-close hooks at 1675785263784Disabling compacts and flushes for region at 1675785263784Disabling writes for close at 1675785263784Writing region close event to WAL at 1675785263809 (+25 ms)Running coprocessor post-close hooks at 1675785263813 (+4 ms)Closed at 1675785263813 2023-02-07 15:54:23,814 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:23,815 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=d49006b752d6e5dc24676c9215eaa7dc, regionState=CLOSED 2023-02-07 15:54:23,815 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785263815"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785263815"}]},"ts":"1675785263815"} 2023-02-07 15:54:23,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-07 15:54:23,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure d49006b752d6e5dc24676c9215eaa7dc, server=jenkins-hbase12.apache.org,38629,1675785236797 in 184 msec 2023-02-07 15:54:23,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-07 15:54:23,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, UNASSIGN in 189 msec 2023-02-07 15:54:23,820 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785263820"}]},"ts":"1675785263820"} 2023-02-07 15:54:23,821 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-07 15:54:23,843 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-07 15:54:23,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 279 msec 2023-02-07 15:54:23,912 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:23,912 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:23,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-07 15:54:23,913 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-07 15:54:23,913 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:23,913 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:23,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testRollbackAndDoubleExecution 2023-02-07 15:54:23,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:23,917 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:23,917 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:23,917 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:23,917 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:23,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-07 15:54:23,921 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:23,923 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/recovered.edits] 2023-02-07 15:54:23,926 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0af5c2ee62fe4e1f9e0fae6f29d94624 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf1/0af5c2ee62fe4e1f9e0fae6f29d94624 2023-02-07 15:54:23,928 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2fcba979b8814dd2a7eec279bc8c8238 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/cf2/2fcba979b8814dd2a7eec279bc8c8238 2023-02-07 15:54:23,931 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/recovered.edits/90.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc/recovered.edits/90.seqid 2023-02-07 15:54:23,931 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackAndDoubleExecution/d49006b752d6e5dc24676c9215eaa7dc 2023-02-07 15:54:23,931 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-07 15:54:23,935 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:23,937 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-07 15:54:23,939 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-07 15:54:23,941 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:23,941 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-07 15:54:23,941 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785263941"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:23,943 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-07 15:54:23,943 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d49006b752d6e5dc24676c9215eaa7dc, NAME => 'testRollbackAndDoubleExecution,,1675785254419.d49006b752d6e5dc24676c9215eaa7dc.', STARTKEY => '', ENDKEY => ''}] 2023-02-07 15:54:23,943 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-07 15:54:23,943 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675785263943"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:23,945 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-07 15:54:23,956 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:54:23,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 42 msec 2023-02-07 15:54:24,018 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:24,018 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:24,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-07 15:54:24,019 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-07 15:54:24,032 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=400 (was 405), OpenFileDescriptor=847 (was 864), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=344 (was 370), ProcessCount=175 (was 173) - ProcessCount LEAK? -, AvailableMemoryMB=3159 (was 3177) 2023-02-07 15:54:24,045 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=400, OpenFileDescriptor=847, MaxFileDescriptor=60000, SystemLoadAverage=344, ProcessCount=175, AvailableMemoryMB=3158 2023-02-07 15:54:24,045 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:24,048 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:24,048 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:24,060 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 15:54:24,060 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 15:54:24,062 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:54:24,064 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 15:54:24,064 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 15:54:24,066 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:24,067 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2 empty. 2023-02-07 15:54:24,067 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:24,067 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-07 15:54:24,077 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-07 15:54:24,078 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3c6c8eb4e621e1f103ed5aa628afa6b2, NAME => 'testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp 2023-02-07 15:54:24,089 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:24,089 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 3c6c8eb4e621e1f103ed5aa628afa6b2, disabling compactions & flushes 2023-02-07 15:54:24,089 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. 2023-02-07 15:54:24,089 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. 2023-02-07 15:54:24,089 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. after waiting 0 ms 2023-02-07 15:54:24,089 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. 2023-02-07 15:54:24,089 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. 2023-02-07 15:54:24,089 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 3c6c8eb4e621e1f103ed5aa628afa6b2: Waiting for close lock at 1675785264089Disabling compacts and flushes for region at 1675785264089Disabling writes for close at 1675785264089Writing region close event to WAL at 1675785264089Closed at 1675785264089 2023-02-07 15:54:24,091 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 15:54:24,092 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675785264092"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785264092"}]},"ts":"1675785264092"} 2023-02-07 15:54:24,093 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 15:54:24,094 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 15:54:24,094 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785264094"}]},"ts":"1675785264094"} 2023-02-07 15:54:24,096 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-07 15:54:24,117 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:54:24,118 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:54:24,118 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:54:24,118 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:54:24,118 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 15:54:24,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3c6c8eb4e621e1f103ed5aa628afa6b2, ASSIGN}] 2023-02-07 15:54:24,120 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3c6c8eb4e621e1f103ed5aa628afa6b2, ASSIGN 2023-02-07 15:54:24,120 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3c6c8eb4e621e1f103ed5aa628afa6b2, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:54:24,238 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 15:54:24,270 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:54:24,271 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=3c6c8eb4e621e1f103ed5aa628afa6b2, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:24,272 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675785264271"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785264271"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785264271"}]},"ts":"1675785264271"} 2023-02-07 15:54:24,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 3c6c8eb4e621e1f103ed5aa628afa6b2, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:24,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. 2023-02-07 15:54:24,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3c6c8eb4e621e1f103ed5aa628afa6b2, NAME => 'testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:54:24,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:24,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:24,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:24,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:24,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:24,431 INFO [StoreOpener-3c6c8eb4e621e1f103ed5aa628afa6b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:24,432 DEBUG [StoreOpener-3c6c8eb4e621e1f103ed5aa628afa6b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2/cf1 2023-02-07 15:54:24,432 DEBUG [StoreOpener-3c6c8eb4e621e1f103ed5aa628afa6b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2/cf1 2023-02-07 15:54:24,433 INFO [StoreOpener-3c6c8eb4e621e1f103ed5aa628afa6b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3c6c8eb4e621e1f103ed5aa628afa6b2 columnFamilyName cf1 2023-02-07 15:54:24,433 INFO [StoreOpener-3c6c8eb4e621e1f103ed5aa628afa6b2-1] regionserver.HStore(310): Store=3c6c8eb4e621e1f103ed5aa628afa6b2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:24,433 INFO [StoreOpener-3c6c8eb4e621e1f103ed5aa628afa6b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:24,435 DEBUG [StoreOpener-3c6c8eb4e621e1f103ed5aa628afa6b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2/cf2 2023-02-07 15:54:24,435 DEBUG [StoreOpener-3c6c8eb4e621e1f103ed5aa628afa6b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2/cf2 2023-02-07 15:54:24,435 INFO [StoreOpener-3c6c8eb4e621e1f103ed5aa628afa6b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3c6c8eb4e621e1f103ed5aa628afa6b2 columnFamilyName cf2 2023-02-07 15:54:24,436 INFO [StoreOpener-3c6c8eb4e621e1f103ed5aa628afa6b2-1] regionserver.HStore(310): Store=3c6c8eb4e621e1f103ed5aa628afa6b2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:24,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:24,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:24,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:24,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:24,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:54:24,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 3c6c8eb4e621e1f103ed5aa628afa6b2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69249603, jitterRate=0.03189949691295624}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:24,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 3c6c8eb4e621e1f103ed5aa628afa6b2: Running coprocessor pre-open hook at 1675785264429Writing region info on filesystem at 1675785264429Initializing all the Stores at 1675785264430 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785264430Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785264430Cleaning up temporary data from old regions at 1675785264438 (+8 ms)Cleaning up detritus from prior splits at 1675785264438Running coprocessor post-open hooks at 1675785264443 (+5 ms)Region opened successfully at 1675785264443 2023-02-07 15:54:24,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2., pid=55, masterSystemTime=1675785264425 2023-02-07 15:54:24,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. 2023-02-07 15:54:24,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. 2023-02-07 15:54:24,447 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=3c6c8eb4e621e1f103ed5aa628afa6b2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:24,447 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675785264447"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785264447"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785264447"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785264447"}]},"ts":"1675785264447"} 2023-02-07 15:54:24,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-07 15:54:24,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 3c6c8eb4e621e1f103ed5aa628afa6b2, server=jenkins-hbase12.apache.org,38629,1675785236797 in 174 msec 2023-02-07 15:54:24,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-07 15:54:24,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3c6c8eb4e621e1f103ed5aa628afa6b2, ASSIGN in 333 msec 2023-02-07 15:54:24,454 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 15:54:24,454 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785264454"}]},"ts":"1675785264454"} 2023-02-07 15:54:24,455 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-07 15:54:24,486 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 15:54:24,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 425 msec 2023-02-07 15:54:24,563 DEBUG [Listener at localhost.localdomain/41965] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:24,564 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:24,565 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:24,565 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:24,565 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:24,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3c6c8eb4e621e1f103ed5aa628afa6b2, UNASSIGN}] 2023-02-07 15:54:24,568 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3c6c8eb4e621e1f103ed5aa628afa6b2, UNASSIGN 2023-02-07 15:54:24,568 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=3c6c8eb4e621e1f103ed5aa628afa6b2, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:24,568 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675785264568"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785264568"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785264568"}]},"ts":"1675785264568"} 2023-02-07 15:54:24,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 3c6c8eb4e621e1f103ed5aa628afa6b2, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:24,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:24,723 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 3c6c8eb4e621e1f103ed5aa628afa6b2, disabling compactions & flushes 2023-02-07 15:54:24,723 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. 2023-02-07 15:54:24,723 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. 2023-02-07 15:54:24,723 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. after waiting 0 ms 2023-02-07 15:54:24,723 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. 2023-02-07 15:54:24,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-07 15:54:24,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:24,727 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2. 2023-02-07 15:54:24,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 3c6c8eb4e621e1f103ed5aa628afa6b2: Waiting for close lock at 1675785264722Running coprocessor pre-close hooks at 1675785264722Disabling compacts and flushes for region at 1675785264722Disabling writes for close at 1675785264723 (+1 ms)Writing region close event to WAL at 1675785264723Running coprocessor post-close hooks at 1675785264727 (+4 ms)Closed at 1675785264727 2023-02-07 15:54:24,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:24,729 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=3c6c8eb4e621e1f103ed5aa628afa6b2, regionState=CLOSED 2023-02-07 15:54:24,729 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675785264729"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785264729"}]},"ts":"1675785264729"} 2023-02-07 15:54:24,732 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-07 15:54:24,732 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 3c6c8eb4e621e1f103ed5aa628afa6b2, server=jenkins-hbase12.apache.org,38629,1675785236797 in 160 msec 2023-02-07 15:54:24,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-07 15:54:24,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3c6c8eb4e621e1f103ed5aa628afa6b2, UNASSIGN in 165 msec 2023-02-07 15:54:24,756 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-07 15:54:24,757 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-07 15:54:24,759 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675785264759"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1675785264759"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1675785264759"}]},"ts":"1675785264759"} 2023-02-07 15:54:24,759 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785264759"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785264759"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785264759"}]},"ts":"1675785264759"} 2023-02-07 15:54:24,759 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785264759"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785264759"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785264759"}]},"ts":"1675785264759"} 2023-02-07 15:54:24,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1bdbb1dd25cc1c811bd033b566adb3f8, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5866b03ffd45bb7a8602b40ca7d13ab0, ASSIGN}] 2023-02-07 15:54:24,763 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5866b03ffd45bb7a8602b40ca7d13ab0, ASSIGN 2023-02-07 15:54:24,764 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5866b03ffd45bb7a8602b40ca7d13ab0, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:54:24,764 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1bdbb1dd25cc1c811bd033b566adb3f8, ASSIGN 2023-02-07 15:54:24,765 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1bdbb1dd25cc1c811bd033b566adb3f8, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:54:24,914 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 15:54:24,915 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=1bdbb1dd25cc1c811bd033b566adb3f8, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:24,915 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785264915"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785264915"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785264915"}]},"ts":"1675785264915"} 2023-02-07 15:54:24,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 1bdbb1dd25cc1c811bd033b566adb3f8, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:24,919 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=5866b03ffd45bb7a8602b40ca7d13ab0, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:24,920 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785264919"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785264919"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785264919"}]},"ts":"1675785264919"} 2023-02-07 15:54:24,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 5866b03ffd45bb7a8602b40ca7d13ab0, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:25,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0. 2023-02-07 15:54:25,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5866b03ffd45bb7a8602b40ca7d13ab0, NAME => 'testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0.', STARTKEY => '41', ENDKEY => ''} 2023-02-07 15:54:25,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:25,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:25,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:25,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:25,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:25,074 INFO [StoreOpener-5866b03ffd45bb7a8602b40ca7d13ab0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:25,075 DEBUG [StoreOpener-5866b03ffd45bb7a8602b40ca7d13ab0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0/cf1 2023-02-07 15:54:25,075 DEBUG [StoreOpener-5866b03ffd45bb7a8602b40ca7d13ab0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0/cf1 2023-02-07 15:54:25,076 INFO [StoreOpener-5866b03ffd45bb7a8602b40ca7d13ab0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5866b03ffd45bb7a8602b40ca7d13ab0 columnFamilyName cf1 2023-02-07 15:54:25,076 INFO [StoreOpener-5866b03ffd45bb7a8602b40ca7d13ab0-1] regionserver.HStore(310): Store=5866b03ffd45bb7a8602b40ca7d13ab0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:25,076 INFO [StoreOpener-5866b03ffd45bb7a8602b40ca7d13ab0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:25,078 DEBUG [StoreOpener-5866b03ffd45bb7a8602b40ca7d13ab0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0/cf2 2023-02-07 15:54:25,078 DEBUG [StoreOpener-5866b03ffd45bb7a8602b40ca7d13ab0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0/cf2 2023-02-07 15:54:25,078 INFO [StoreOpener-5866b03ffd45bb7a8602b40ca7d13ab0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5866b03ffd45bb7a8602b40ca7d13ab0 columnFamilyName cf2 2023-02-07 15:54:25,079 INFO [StoreOpener-5866b03ffd45bb7a8602b40ca7d13ab0-1] regionserver.HStore(310): Store=5866b03ffd45bb7a8602b40ca7d13ab0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:25,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:25,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:25,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:25,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:25,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 5866b03ffd45bb7a8602b40ca7d13ab0; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69558649, jitterRate=0.03650464117527008}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:25,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 5866b03ffd45bb7a8602b40ca7d13ab0: Running coprocessor pre-open hook at 1675785265073Writing region info on filesystem at 1675785265073Initializing all the Stores at 1675785265073Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785265073Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785265074 (+1 ms)Cleaning up temporary data from old regions at 1675785265081 (+7 ms)Cleaning up detritus from prior splits at 1675785265082 (+1 ms)Running coprocessor post-open hooks at 1675785265085 (+3 ms)Region opened successfully at 1675785265085 2023-02-07 15:54:25,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0., pid=62, masterSystemTime=1675785265069 2023-02-07 15:54:25,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0. 2023-02-07 15:54:25,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0. 2023-02-07 15:54:25,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8. 2023-02-07 15:54:25,088 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=5866b03ffd45bb7a8602b40ca7d13ab0, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:25,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1bdbb1dd25cc1c811bd033b566adb3f8, NAME => 'testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8.', STARTKEY => '', ENDKEY => '41'} 2023-02-07 15:54:25,088 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785265088"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785265088"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785265088"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785265088"}]},"ts":"1675785265088"} 2023-02-07 15:54:25,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:25,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 1bdbb1dd25cc1c811bd033b566adb3f8 2023-02-07 15:54:25,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:25,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 1bdbb1dd25cc1c811bd033b566adb3f8 2023-02-07 15:54:25,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 1bdbb1dd25cc1c811bd033b566adb3f8 2023-02-07 15:54:25,090 INFO [StoreOpener-1bdbb1dd25cc1c811bd033b566adb3f8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1bdbb1dd25cc1c811bd033b566adb3f8 2023-02-07 15:54:25,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-07 15:54:25,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 5866b03ffd45bb7a8602b40ca7d13ab0, server=jenkins-hbase12.apache.org,38629,1675785236797 in 169 msec 2023-02-07 15:54:25,092 DEBUG [StoreOpener-1bdbb1dd25cc1c811bd033b566adb3f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8/cf1 2023-02-07 15:54:25,092 DEBUG [StoreOpener-1bdbb1dd25cc1c811bd033b566adb3f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8/cf1 2023-02-07 15:54:25,092 INFO [StoreOpener-1bdbb1dd25cc1c811bd033b566adb3f8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1bdbb1dd25cc1c811bd033b566adb3f8 columnFamilyName cf1 2023-02-07 15:54:25,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5866b03ffd45bb7a8602b40ca7d13ab0, ASSIGN in 329 msec 2023-02-07 15:54:25,093 INFO [StoreOpener-1bdbb1dd25cc1c811bd033b566adb3f8-1] regionserver.HStore(310): Store=1bdbb1dd25cc1c811bd033b566adb3f8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:25,093 INFO [StoreOpener-1bdbb1dd25cc1c811bd033b566adb3f8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1bdbb1dd25cc1c811bd033b566adb3f8 2023-02-07 15:54:25,094 DEBUG [StoreOpener-1bdbb1dd25cc1c811bd033b566adb3f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8/cf2 2023-02-07 15:54:25,094 DEBUG [StoreOpener-1bdbb1dd25cc1c811bd033b566adb3f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8/cf2 2023-02-07 15:54:25,094 INFO [StoreOpener-1bdbb1dd25cc1c811bd033b566adb3f8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1bdbb1dd25cc1c811bd033b566adb3f8 columnFamilyName cf2 2023-02-07 15:54:25,095 INFO [StoreOpener-1bdbb1dd25cc1c811bd033b566adb3f8-1] regionserver.HStore(310): Store=1bdbb1dd25cc1c811bd033b566adb3f8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:25,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8 2023-02-07 15:54:25,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8 2023-02-07 15:54:25,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:25,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 1bdbb1dd25cc1c811bd033b566adb3f8 2023-02-07 15:54:25,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 1bdbb1dd25cc1c811bd033b566adb3f8; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68835958, jitterRate=0.025735706090927124}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:25,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 1bdbb1dd25cc1c811bd033b566adb3f8: Running coprocessor pre-open hook at 1675785265089Writing region info on filesystem at 1675785265089Initializing all the Stores at 1675785265090 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785265090Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785265090Cleaning up temporary data from old regions at 1675785265097 (+7 ms)Cleaning up detritus from prior splits at 1675785265098 (+1 ms)Running coprocessor post-open hooks at 1675785265100 (+2 ms)Region opened successfully at 1675785265100 2023-02-07 15:54:25,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8., pid=61, masterSystemTime=1675785265069 2023-02-07 15:54:25,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8. 2023-02-07 15:54:25,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8. 2023-02-07 15:54:25,103 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=1bdbb1dd25cc1c811bd033b566adb3f8, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:25,103 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785265103"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785265103"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785265103"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785265103"}]},"ts":"1675785265103"} 2023-02-07 15:54:25,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-07 15:54:25,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 1bdbb1dd25cc1c811bd033b566adb3f8, server=jenkins-hbase12.apache.org,38629,1675785236797 in 188 msec 2023-02-07 15:54:25,108 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-07 15:54:25,108 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1bdbb1dd25cc1c811bd033b566adb3f8, ASSIGN in 345 msec 2023-02-07 15:54:25,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 in 545 msec 2023-02-07 15:54:25,319 DEBUG [Listener at localhost.localdomain/41965] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 15:54:25,323 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:25,323 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:25,323 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:25,324 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:25,324 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:25,324 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-07 15:54:25,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegionNoStoreFile 2023-02-07 15:54:25,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:54:25,327 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:25,327 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:25,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-07 15:54:25,328 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785265328"}]},"ts":"1675785265328"} 2023-02-07 15:54:25,329 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-07 15:54:25,397 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-07 15:54:25,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1bdbb1dd25cc1c811bd033b566adb3f8, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5866b03ffd45bb7a8602b40ca7d13ab0, UNASSIGN}] 2023-02-07 15:54:25,398 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5866b03ffd45bb7a8602b40ca7d13ab0, UNASSIGN 2023-02-07 15:54:25,399 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=5866b03ffd45bb7a8602b40ca7d13ab0, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:25,399 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785265399"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785265399"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785265399"}]},"ts":"1675785265399"} 2023-02-07 15:54:25,400 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 5866b03ffd45bb7a8602b40ca7d13ab0, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:25,403 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1bdbb1dd25cc1c811bd033b566adb3f8, UNASSIGN 2023-02-07 15:54:25,403 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=1bdbb1dd25cc1c811bd033b566adb3f8, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:25,403 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785265403"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785265403"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785265403"}]},"ts":"1675785265403"} 2023-02-07 15:54:25,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 1bdbb1dd25cc1c811bd033b566adb3f8, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:25,428 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:25,428 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:25,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-07 15:54:25,552 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 1bdbb1dd25cc1c811bd033b566adb3f8 2023-02-07 15:54:25,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 1bdbb1dd25cc1c811bd033b566adb3f8, disabling compactions & flushes 2023-02-07 15:54:25,554 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8. 2023-02-07 15:54:25,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8. 2023-02-07 15:54:25,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8. after waiting 0 ms 2023-02-07 15:54:25,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8. 2023-02-07 15:54:25,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-07 15:54:25,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:25,559 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8. 2023-02-07 15:54:25,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 1bdbb1dd25cc1c811bd033b566adb3f8: Waiting for close lock at 1675785265553Running coprocessor pre-close hooks at 1675785265553Disabling compacts and flushes for region at 1675785265553Disabling writes for close at 1675785265554 (+1 ms)Writing region close event to WAL at 1675785265554Running coprocessor post-close hooks at 1675785265559 (+5 ms)Closed at 1675785265559 2023-02-07 15:54:25,560 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 1bdbb1dd25cc1c811bd033b566adb3f8 2023-02-07 15:54:25,561 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:25,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 5866b03ffd45bb7a8602b40ca7d13ab0, disabling compactions & flushes 2023-02-07 15:54:25,561 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0. 2023-02-07 15:54:25,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0. 2023-02-07 15:54:25,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0. after waiting 0 ms 2023-02-07 15:54:25,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0. 2023-02-07 15:54:25,562 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=1bdbb1dd25cc1c811bd033b566adb3f8, regionState=CLOSED 2023-02-07 15:54:25,562 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785265562"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785265562"}]},"ts":"1675785265562"} 2023-02-07 15:54:25,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-07 15:54:25,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-07 15:54:25,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 1bdbb1dd25cc1c811bd033b566adb3f8, server=jenkins-hbase12.apache.org,38629,1675785236797 in 159 msec 2023-02-07 15:54:25,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:25,566 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0. 2023-02-07 15:54:25,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 5866b03ffd45bb7a8602b40ca7d13ab0: Waiting for close lock at 1675785265561Running coprocessor pre-close hooks at 1675785265561Disabling compacts and flushes for region at 1675785265561Disabling writes for close at 1675785265562 (+1 ms)Writing region close event to WAL at 1675785265562Running coprocessor post-close hooks at 1675785265566 (+4 ms)Closed at 1675785265566 2023-02-07 15:54:25,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1bdbb1dd25cc1c811bd033b566adb3f8, UNASSIGN in 168 msec 2023-02-07 15:54:25,567 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:25,568 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=5866b03ffd45bb7a8602b40ca7d13ab0, regionState=CLOSED 2023-02-07 15:54:25,568 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785265568"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785265568"}]},"ts":"1675785265568"} 2023-02-07 15:54:25,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-07 15:54:25,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 5866b03ffd45bb7a8602b40ca7d13ab0, server=jenkins-hbase12.apache.org,38629,1675785236797 in 169 msec 2023-02-07 15:54:25,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-02-07 15:54:25,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5866b03ffd45bb7a8602b40ca7d13ab0, UNASSIGN in 173 msec 2023-02-07 15:54:25,574 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785265573"}]},"ts":"1675785265573"} 2023-02-07 15:54:25,575 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-07 15:54:25,586 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-07 15:54:25,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 262 msec 2023-02-07 15:54:25,630 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:25,630 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:25,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-07 15:54:25,631 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-07 15:54:25,631 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:25,631 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:25,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegionNoStoreFile 2023-02-07 15:54:25,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:54:25,634 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:54:25,635 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:25,635 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:25,635 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:54:25,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-07 15:54:25,640 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:25,640 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:25,640 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8 2023-02-07 15:54:25,642 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8/recovered.edits] 2023-02-07 15:54:25,642 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0/recovered.edits] 2023-02-07 15:54:25,642 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2/recovered.edits] 2023-02-07 15:54:25,648 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2/recovered.edits/4.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2/recovered.edits/4.seqid 2023-02-07 15:54:25,649 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0/recovered.edits/7.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0/recovered.edits/7.seqid 2023-02-07 15:54:25,649 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8/recovered.edits/7.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8/recovered.edits/7.seqid 2023-02-07 15:54:25,649 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/3c6c8eb4e621e1f103ed5aa628afa6b2 2023-02-07 15:54:25,649 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:54:25,649 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionNoStoreFile/1bdbb1dd25cc1c811bd033b566adb3f8 2023-02-07 15:54:25,649 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-07 15:54:25,652 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:54:25,654 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-07 15:54:25,656 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-07 15:54:25,657 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:54:25,657 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-07 15:54:25,657 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785265657"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:25,658 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785265657"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:25,658 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785265657"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:25,660 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 15:54:25,660 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3c6c8eb4e621e1f103ed5aa628afa6b2, NAME => 'testSplitTableRegionNoStoreFile,,1675785264061.3c6c8eb4e621e1f103ed5aa628afa6b2.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 1bdbb1dd25cc1c811bd033b566adb3f8, NAME => 'testSplitTableRegionNoStoreFile,,1675785264563.1bdbb1dd25cc1c811bd033b566adb3f8.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 5866b03ffd45bb7a8602b40ca7d13ab0, NAME => 'testSplitTableRegionNoStoreFile,41,1675785264563.5866b03ffd45bb7a8602b40ca7d13ab0.', STARTKEY => '41', ENDKEY => ''}] 2023-02-07 15:54:25,660 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-07 15:54:25,660 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675785265660"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:25,662 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-07 15:54:25,675 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:54:25,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 43 msec 2023-02-07 15:54:25,736 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:25,736 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:25,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-07 15:54:25,738 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-07 15:54:25,752 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=404 (was 400) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_846220023_15 at /127.0.0.1:53010 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_846220023_15 at /127.0.0.1:33550 [Waiting for operation #52] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=847 (was 847), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=324 (was 344), ProcessCount=173 (was 175), AvailableMemoryMB=3146 (was 3158) 2023-02-07 15:54:25,765 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=404, OpenFileDescriptor=847, MaxFileDescriptor=60000, SystemLoadAverage=324, ProcessCount=173, AvailableMemoryMB=3145 2023-02-07 15:54:25,766 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:25,766 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:25,766 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:25,779 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 15:54:25,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 15:54:25,781 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:25,784 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 15:54:25,785 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 15:54:25,787 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:25,788 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344 empty. 2023-02-07 15:54:25,788 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:25,788 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-07 15:54:25,799 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-07 15:54:25,801 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 33148689d0ba6f95ebbfc74509e65344, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp 2023-02-07 15:54:25,818 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:25,818 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 33148689d0ba6f95ebbfc74509e65344, disabling compactions & flushes 2023-02-07 15:54:25,818 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:25,818 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:25,818 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. after waiting 0 ms 2023-02-07 15:54:25,818 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:25,819 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:25,819 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 33148689d0ba6f95ebbfc74509e65344: Waiting for close lock at 1675785265818Disabling compacts and flushes for region at 1675785265818Disabling writes for close at 1675785265818Writing region close event to WAL at 1675785265819 (+1 ms)Closed at 1675785265819 2023-02-07 15:54:25,822 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 15:54:25,823 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675785265822"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785265822"}]},"ts":"1675785265822"} 2023-02-07 15:54:25,824 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 15:54:25,825 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 15:54:25,825 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785265825"}]},"ts":"1675785265825"} 2023-02-07 15:54:25,827 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-07 15:54:25,850 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:54:25,851 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:54:25,851 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:54:25,851 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:54:25,851 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 15:54:25,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=33148689d0ba6f95ebbfc74509e65344, ASSIGN}] 2023-02-07 15:54:25,854 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=33148689d0ba6f95ebbfc74509e65344, ASSIGN 2023-02-07 15:54:25,855 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=33148689d0ba6f95ebbfc74509e65344, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:54:26,005 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:54:26,006 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=33148689d0ba6f95ebbfc74509e65344, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,006 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675785266006"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785266006"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785266006"}]},"ts":"1675785266006"} 2023-02-07 15:54:26,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 33148689d0ba6f95ebbfc74509e65344, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:26,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:26,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 33148689d0ba6f95ebbfc74509e65344, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:54:26,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:26,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:26,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:26,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:26,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:26,165 INFO [StoreOpener-33148689d0ba6f95ebbfc74509e65344-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:26,167 DEBUG [StoreOpener-33148689d0ba6f95ebbfc74509e65344-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1 2023-02-07 15:54:26,167 DEBUG [StoreOpener-33148689d0ba6f95ebbfc74509e65344-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1 2023-02-07 15:54:26,167 INFO [StoreOpener-33148689d0ba6f95ebbfc74509e65344-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 33148689d0ba6f95ebbfc74509e65344 columnFamilyName cf1 2023-02-07 15:54:26,168 INFO [StoreOpener-33148689d0ba6f95ebbfc74509e65344-1] regionserver.HStore(310): Store=33148689d0ba6f95ebbfc74509e65344/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:26,168 INFO [StoreOpener-33148689d0ba6f95ebbfc74509e65344-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:26,169 DEBUG [StoreOpener-33148689d0ba6f95ebbfc74509e65344-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2 2023-02-07 15:54:26,169 DEBUG [StoreOpener-33148689d0ba6f95ebbfc74509e65344-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2 2023-02-07 15:54:26,170 INFO [StoreOpener-33148689d0ba6f95ebbfc74509e65344-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 33148689d0ba6f95ebbfc74509e65344 columnFamilyName cf2 2023-02-07 15:54:26,170 INFO [StoreOpener-33148689d0ba6f95ebbfc74509e65344-1] regionserver.HStore(310): Store=33148689d0ba6f95ebbfc74509e65344/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:26,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:26,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:26,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:26,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:26,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:54:26,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 33148689d0ba6f95ebbfc74509e65344; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65425307, jitterRate=-0.02508695423603058}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:26,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 33148689d0ba6f95ebbfc74509e65344: Running coprocessor pre-open hook at 1675785266164Writing region info on filesystem at 1675785266164Initializing all the Stores at 1675785266165 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785266165Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785266165Cleaning up temporary data from old regions at 1675785266172 (+7 ms)Cleaning up detritus from prior splits at 1675785266172Running coprocessor post-open hooks at 1675785266176 (+4 ms)Region opened successfully at 1675785266176 2023-02-07 15:54:26,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344., pid=71, masterSystemTime=1675785266160 2023-02-07 15:54:26,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:26,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:26,179 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=33148689d0ba6f95ebbfc74509e65344, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,179 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675785266179"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785266179"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785266179"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785266179"}]},"ts":"1675785266179"} 2023-02-07 15:54:26,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-07 15:54:26,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 33148689d0ba6f95ebbfc74509e65344, server=jenkins-hbase12.apache.org,38629,1675785236797 in 172 msec 2023-02-07 15:54:26,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-07 15:54:26,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=33148689d0ba6f95ebbfc74509e65344, ASSIGN in 330 msec 2023-02-07 15:54:26,184 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 15:54:26,184 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785266184"}]},"ts":"1675785266184"} 2023-02-07 15:54:26,187 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-07 15:54:26,254 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 15:54:26,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 474 msec 2023-02-07 15:54:26,291 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:26,291 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:26,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:26,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,294 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:26,294 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:26,294 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 15:54:26,294 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 15:54:26,294 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,295 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,306 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,306 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,306 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,306 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,306 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,306 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:26,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,306 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,307 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:26,307 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:26,307 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:26,307 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,307 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:26,307 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:26,308 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:26,307 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:26,308 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:26,308 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:26,308 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:26,308 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:26,308 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:26,308 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:26,308 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:26,308 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:26,308 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:26,308 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. started... 2023-02-07 15:54:26,309 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 33148689d0ba6f95ebbfc74509e65344 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 15:54:26,316 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,316 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,316 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,316 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,316 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:26,316 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:26,326 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/862a9db3a85944f984190fba7572a5ae 2023-02-07 15:54:26,340 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f 2023-02-07 15:54:26,344 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/862a9db3a85944f984190fba7572a5ae as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/862a9db3a85944f984190fba7572a5ae 2023-02-07 15:54:26,349 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/862a9db3a85944f984190fba7572a5ae, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:54:26,350 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f 2023-02-07 15:54:26,354 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:54:26,355 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 33148689d0ba6f95ebbfc74509e65344 in 46ms, sequenceid=6, compaction requested=false 2023-02-07 15:54:26,355 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:26,356 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 33148689d0ba6f95ebbfc74509e65344: 2023-02-07 15:54:26,356 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:26,356 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:26,356 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:26,356 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,356 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:26,356 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:26,369 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,369 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:26,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:26,369 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,369 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:26,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:26,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 15:54:26,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b885e6c[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:26,371 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:26,371 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,379 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,379 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,379 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,380 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,380 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:26,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,380 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:26,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,380 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:26,380 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:26,380 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:26,380 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:26,380 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:26,380 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:26,380 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:26,380 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:26,390 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:26,390 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:26,390 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:26,400 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:26,400 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,400 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:26,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:26,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:26,401 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:26,401 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:26,401 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:26,401 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:26,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:26,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:26,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:26,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:26,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:26,404 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 15:54:26,404 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:26,404 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:26,404 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,404 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:26,411 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,411 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,411 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:26,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,411 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,411 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,411 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,411 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,411 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:26,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:26,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:26,412 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,412 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:26,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,413 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:26,414 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,414 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,447 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,447 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,447 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,447 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:26,447 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,447 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,447 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,447 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:26,447 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:26,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:26,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:26,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,447 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,447 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:26,448 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,448 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,448 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,448 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,448 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,448 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:26,448 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:26,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:26,448 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:26,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:26,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:26,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:26,448 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:26,548 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:26,548 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:26,548 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:26,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:26,563 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:26,563 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:26,564 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:26,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:26,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,565 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:26,565 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:26,565 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 15:54:26,565 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 15:54:26,566 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,566 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,586 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,586 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,586 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,586 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,587 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,587 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,587 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:26,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,588 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:26,588 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:26,588 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,588 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,588 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:26,588 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:26,588 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:26,588 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:26,589 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. started... 2023-02-07 15:54:26,589 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 33148689d0ba6f95ebbfc74509e65344 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:26,600 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,600 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,601 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,601 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,601 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:26,601 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:26,603 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/3177174266594d6cb746e113a8861a10 2023-02-07 15:54:26,622 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/5cf1b97a9fd14df985aeee1a531d1227 2023-02-07 15:54:26,627 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/3177174266594d6cb746e113a8861a10 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/3177174266594d6cb746e113a8861a10 2023-02-07 15:54:26,631 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/3177174266594d6cb746e113a8861a10, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:54:26,632 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/5cf1b97a9fd14df985aeee1a531d1227 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/5cf1b97a9fd14df985aeee1a531d1227 2023-02-07 15:54:26,636 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/5cf1b97a9fd14df985aeee1a531d1227, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:54:26,637 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 33148689d0ba6f95ebbfc74509e65344 in 48ms, sequenceid=19, compaction requested=false 2023-02-07 15:54:26,637 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 33148689d0ba6f95ebbfc74509e65344: 2023-02-07 15:54:26,637 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:26,637 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:26,637 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:26,637 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,637 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:26,637 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:26,649 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,649 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:26,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:26,650 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,650 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:26,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:26,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:26,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:26,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 15:54:26,652 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:26,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58d8b96a[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:26,652 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,664 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,664 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,664 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,664 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,664 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:26,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,664 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:26,664 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:26,664 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:26,664 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:26,664 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:26,664 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:26,664 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:26,664 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:26,664 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:26,674 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:26,674 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:26,674 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:26,685 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,685 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:26,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:26,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:26,685 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:26,685 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:26,685 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:26,685 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:26,685 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:26,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:26,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:26,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:26,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:26,688 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 15:54:26,688 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:26,688 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:26,688 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,688 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:26,695 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,695 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,695 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,695 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:26,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,695 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:26,695 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:26,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:26,696 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:26,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:26,696 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:26,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:26,696 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:26,697 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:26,698 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,698 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,698 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:26,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:26,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:26,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:26,706 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:26,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:26,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:26,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,707 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:26,707 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:26,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:26,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:26,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:26,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:26,707 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:26,707 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:26,707 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:26,707 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,707 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,707 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,807 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:26,808 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:26,808 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:26,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:26,820 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:26,820 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:26,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:26,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:26,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,822 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:26,822 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:26,822 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 15:54:26,822 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 15:54:26,823 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,823 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,832 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,832 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,832 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,832 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:26,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,832 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:26,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:26,832 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:26,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:26,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:26,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,833 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,834 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:26,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:26,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:26,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,834 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,834 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:26,834 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,834 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:26,834 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:26,834 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:26,834 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:26,834 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:26,834 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:26,835 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. started... 2023-02-07 15:54:26,835 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 33148689d0ba6f95ebbfc74509e65344 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:26,871 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,891 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,891 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,891 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:26,892 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:26,892 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:27,248 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/46e65271869541d3aaab446dfe308ecf 2023-02-07 15:54:27,261 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/bc68958b9c3c4bc79a465a41c2db5fdf 2023-02-07 15:54:27,286 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/46e65271869541d3aaab446dfe308ecf as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/46e65271869541d3aaab446dfe308ecf 2023-02-07 15:54:27,290 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/46e65271869541d3aaab446dfe308ecf, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:54:27,291 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/bc68958b9c3c4bc79a465a41c2db5fdf as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/bc68958b9c3c4bc79a465a41c2db5fdf 2023-02-07 15:54:27,296 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/bc68958b9c3c4bc79a465a41c2db5fdf, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:54:27,296 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 33148689d0ba6f95ebbfc74509e65344 in 461ms, sequenceid=32, compaction requested=true 2023-02-07 15:54:27,296 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 33148689d0ba6f95ebbfc74509e65344: 2023-02-07 15:54:27,296 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:27,297 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:27,297 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:27,297 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,297 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:27,297 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:27,348 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,348 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:27,348 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,348 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:27,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 15:54:27,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c9c93a[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:27,350 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:27,350 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,358 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,358 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,358 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,359 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,359 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:27,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,359 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:27,359 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,359 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:27,359 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:27,359 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:27,359 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:27,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,359 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:27,359 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:27,359 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:27,369 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:27,369 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:27,369 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:27,379 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:27,379 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,379 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:27,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:27,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:27,380 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:27,380 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:27,380 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:27,380 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:27,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:27,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:27,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:27,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:27,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:27,383 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 15:54:27,383 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:27,383 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:27,383 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,383 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:27,390 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,390 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,390 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,390 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,390 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,390 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,391 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,390 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:27,391 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,391 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:27,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,392 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:27,393 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,393 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,400 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,400 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,400 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,400 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,401 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,401 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:27,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:27,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:27,401 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,400 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,401 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:27,400 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,401 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,401 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:27,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,401 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,402 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,402 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,402 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,402 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,402 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,402 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,402 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,502 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:27,502 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:27,502 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:27,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:27,513 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:27,513 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:27,514 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:27,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:27,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,515 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:27,515 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:27,515 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 15:54:27,515 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 15:54:27,516 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,516 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,527 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,527 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,527 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,527 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,527 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,527 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:27,528 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:27,528 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:27,529 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,528 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:27,529 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:27,529 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:27,529 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. started... 2023-02-07 15:54:27,529 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 33148689d0ba6f95ebbfc74509e65344 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:27,537 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,537 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,537 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/6d35e862e6ae4513b97ed6281ec5e495 2023-02-07 15:54:27,537 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,538 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:27,538 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,538 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:27,548 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/55a5b4e723b54362ad08f7c6c952ac2f 2023-02-07 15:54:27,552 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/6d35e862e6ae4513b97ed6281ec5e495 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6d35e862e6ae4513b97ed6281ec5e495 2023-02-07 15:54:27,556 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6d35e862e6ae4513b97ed6281ec5e495, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:54:27,556 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/55a5b4e723b54362ad08f7c6c952ac2f as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/55a5b4e723b54362ad08f7c6c952ac2f 2023-02-07 15:54:27,561 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/55a5b4e723b54362ad08f7c6c952ac2f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:54:27,562 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 33148689d0ba6f95ebbfc74509e65344 in 33ms, sequenceid=45, compaction requested=true 2023-02-07 15:54:27,562 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 33148689d0ba6f95ebbfc74509e65344: 2023-02-07 15:54:27,563 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:27,563 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:27,563 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:27,563 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,563 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:27,563 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:27,576 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,576 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:27,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:27,576 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,576 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:27,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:27,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:27,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:27,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 15:54:27,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f8f35e[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:27,578 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:27,578 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,590 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,590 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,590 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,590 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,590 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:27,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,590 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:27,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,590 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:27,591 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:27,590 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:27,591 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:27,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,591 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:27,591 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:27,591 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:27,591 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:27,600 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:27,600 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,600 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:27,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:27,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:27,601 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:27,601 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:27,601 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:27,601 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:27,641 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:27,641 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:27,641 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:27,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:27,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:27,644 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 15:54:27,644 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:27,644 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:27,644 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,644 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:27,664 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,664 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,664 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,664 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,664 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:27,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,664 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,664 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:27,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,664 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,665 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:27,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,665 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:27,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,666 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:27,667 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,667 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,674 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,674 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,674 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,674 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,674 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,675 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:27,675 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:27,674 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:27,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,674 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,674 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,674 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,675 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:27,675 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,675 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,675 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,675 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,675 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,676 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,676 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,676 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,676 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,676 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,776 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:27,776 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:27,776 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:27,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:27,789 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:27,789 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:27,790 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:27,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:27,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,792 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:27,792 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:27,792 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 15:54:27,792 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 15:54:27,792 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,792 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,835 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,835 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,835 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,835 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,836 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,836 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:27,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,836 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,836 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,836 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:27,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:27,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,837 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:27,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,837 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,837 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,837 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:27,837 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,837 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:27,837 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:27,837 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:27,837 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:27,837 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:27,841 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:27,841 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:27,841 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,841 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,841 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:27,841 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:27,841 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:27,841 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:27,841 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:27,841 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:27,841 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:27,841 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:27,841 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:27,841 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:27,841 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:27,841 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. started... 2023-02-07 15:54:27,842 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 33148689d0ba6f95ebbfc74509e65344 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:27,851 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/c67fb23ee9dd4c79a232cde1e8b8a0e0 2023-02-07 15:54:27,853 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,853 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,853 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:27,862 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/c585b51e95ad4dbea65db42982d7d7e7 2023-02-07 15:54:27,863 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,864 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,864 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:27,866 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/c67fb23ee9dd4c79a232cde1e8b8a0e0 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/c67fb23ee9dd4c79a232cde1e8b8a0e0 2023-02-07 15:54:27,869 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/c67fb23ee9dd4c79a232cde1e8b8a0e0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:54:27,870 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/c585b51e95ad4dbea65db42982d7d7e7 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/c585b51e95ad4dbea65db42982d7d7e7 2023-02-07 15:54:27,874 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/c585b51e95ad4dbea65db42982d7d7e7, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:54:27,875 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 33148689d0ba6f95ebbfc74509e65344 in 33ms, sequenceid=58, compaction requested=true 2023-02-07 15:54:27,875 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 33148689d0ba6f95ebbfc74509e65344: 2023-02-07 15:54:27,875 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:27,875 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:27,875 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:27,875 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,875 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:27,875 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:27,885 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,885 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:27,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:27,885 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:27,885 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:27,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:27,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:27,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 15:54:27,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@90ab613[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:27,887 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:27,887 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,895 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,895 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,895 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,895 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,896 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:27,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,896 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:27,896 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:27,896 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:27,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,896 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:27,896 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:27,896 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:27,896 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:27,896 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:27,896 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:27,906 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:27,906 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:27,906 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:27,916 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:27,916 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,916 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:27,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:27,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:27,916 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:27,927 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:27,927 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:27,927 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:27,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:27,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:27,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:27,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:27,930 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 15:54:27,930 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:27,930 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:27,930 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,930 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:27,937 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,937 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,937 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,937 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,937 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,938 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,937 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,938 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:27,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,938 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,938 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,940 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,940 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,940 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:27,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:27,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,980 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:27,980 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:27,980 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:27,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:27,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:27,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:27,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:27,981 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:27,981 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,981 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:27,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,981 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:27,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:27,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,981 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:27,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:27,981 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:27,981 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,081 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:28,081 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:28,081 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:28,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:28,095 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:28,095 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:28,096 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:28,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:28,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,098 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:28,098 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,098 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 15:54:28,098 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 15:54:28,098 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,098 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,116 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,116 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,116 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,116 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,117 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,117 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:28,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,117 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,117 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,117 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:28,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:28,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:28,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,118 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,118 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,118 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,118 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:28,118 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,118 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:28,118 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:28,118 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:28,118 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,118 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,118 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,118 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:28,118 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:28,118 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,118 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:28,118 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:28,118 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:28,118 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,119 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:28,119 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:28,119 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:28,119 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:28,119 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:28,119 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:28,119 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:28,119 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. started... 2023-02-07 15:54:28,119 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 33148689d0ba6f95ebbfc74509e65344 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:54:28,126 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/bd94f04f7ae04647b0a1056004c41a1b 2023-02-07 15:54:28,127 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,136 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/d7a12805559841128b429a0dde6f1386 2023-02-07 15:54:28,137 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,137 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,137 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:28,138 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,138 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:28,140 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/bd94f04f7ae04647b0a1056004c41a1b as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/bd94f04f7ae04647b0a1056004c41a1b 2023-02-07 15:54:28,144 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/bd94f04f7ae04647b0a1056004c41a1b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:54:28,145 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/d7a12805559841128b429a0dde6f1386 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/d7a12805559841128b429a0dde6f1386 2023-02-07 15:54:28,149 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/d7a12805559841128b429a0dde6f1386, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:54:28,149 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 33148689d0ba6f95ebbfc74509e65344 in 30ms, sequenceid=71, compaction requested=true 2023-02-07 15:54:28,150 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 33148689d0ba6f95ebbfc74509e65344: 2023-02-07 15:54:28,150 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:28,150 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:28,150 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:28,150 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,150 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:28,150 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:28,158 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,158 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:28,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:28,159 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,159 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:28,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:28,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:28,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:28,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 15:54:28,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14845db1[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:28,161 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:28,161 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,169 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,169 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,169 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,169 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:28,169 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:28,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,170 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:28,170 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:28,170 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:28,170 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:28,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,169 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,170 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:28,170 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:28,170 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:28,170 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:28,179 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:28,179 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:28,179 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,180 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,180 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:28,180 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:28,190 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:28,190 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,190 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:28,190 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:28,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:28,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:28,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:28,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:28,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:28,193 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:28,193 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 15:54:28,194 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:28,194 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:28,194 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,194 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:28,200 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,200 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,200 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,200 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,201 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:28,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,201 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,200 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,201 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,201 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,202 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,203 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:28,203 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,203 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,211 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,211 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,211 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,211 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,211 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:28,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:28,211 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,211 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,211 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,212 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:28,211 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,212 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,212 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:28,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,212 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,212 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,212 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,212 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,212 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,212 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,212 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,212 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,312 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:28,312 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:28,312 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:28,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:28,326 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:28,326 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:28,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:28,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:28,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,328 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:28,328 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,328 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 15:54:28,328 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 15:54:28,329 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,329 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,337 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,337 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,337 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,337 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,338 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,338 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:28,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:28,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:28,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,339 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:28,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,339 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,339 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,339 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:28,339 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:28,339 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:28,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:28,339 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:28,340 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:28,340 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:28,340 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:28,340 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. started... 2023-02-07 15:54:28,340 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 33148689d0ba6f95ebbfc74509e65344 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-07 15:54:28,348 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,348 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,348 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,348 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,348 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:28,348 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:28,355 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/5f39c7b29bfc4ef596abb57bc6cddb61 2023-02-07 15:54:28,359 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f39c7b29bfc4ef596abb57bc6cddb61 2023-02-07 15:54:28,371 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/1f9a82973f3441019b15a1729eccee63 2023-02-07 15:54:28,375 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f9a82973f3441019b15a1729eccee63 2023-02-07 15:54:28,376 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/5f39c7b29bfc4ef596abb57bc6cddb61 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5f39c7b29bfc4ef596abb57bc6cddb61 2023-02-07 15:54:28,380 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f39c7b29bfc4ef596abb57bc6cddb61 2023-02-07 15:54:28,380 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5f39c7b29bfc4ef596abb57bc6cddb61, entries=9, sequenceid=83, filesize=5.1 K 2023-02-07 15:54:28,381 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/1f9a82973f3441019b15a1729eccee63 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/1f9a82973f3441019b15a1729eccee63 2023-02-07 15:54:28,384 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f9a82973f3441019b15a1729eccee63 2023-02-07 15:54:28,385 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/1f9a82973f3441019b15a1729eccee63, entries=9, sequenceid=83, filesize=5.1 K 2023-02-07 15:54:28,385 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 33148689d0ba6f95ebbfc74509e65344 in 45ms, sequenceid=83, compaction requested=true 2023-02-07 15:54:28,385 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 33148689d0ba6f95ebbfc74509e65344: 2023-02-07 15:54:28,385 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:28,385 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:28,386 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:28,386 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,386 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:28,386 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:28,397 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,397 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:28,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:28,397 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,397 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:28,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:28,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:28,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:28,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 15:54:28,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e4ec602[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:28,399 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:28,399 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,411 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,411 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,411 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,411 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,411 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:28,411 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:28,412 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:28,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,412 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:28,412 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:28,412 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:28,411 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:28,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,412 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:28,412 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:28,412 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:28,444 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:28,444 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,444 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:28,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:28,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:28,444 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,444 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,445 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:28,445 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:28,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:28,464 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:28,464 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,464 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:28,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:28,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:28,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:28,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:28,467 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 15:54:28,467 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:28,467 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:28,467 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,467 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:28,474 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,474 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,474 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,474 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,474 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:28,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:28,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,474 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,474 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:28,475 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,475 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,475 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:28,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:28,476 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,477 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,477 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,485 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:28,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:28,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:28,485 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,486 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,486 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,486 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,486 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,586 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:28,586 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:28,586 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:28,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:28,599 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:28,599 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:28,600 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:28,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:28,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,602 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:28,602 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,602 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 15:54:28,602 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 15:54:28,602 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,602 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,611 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,611 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,611 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,611 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,612 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,612 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,613 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:28,613 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:28,613 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:28,613 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,613 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:28,613 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:28,613 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:28,613 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:28,613 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. started... 2023-02-07 15:54:28,614 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 33148689d0ba6f95ebbfc74509e65344 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-07 15:54:28,621 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,621 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,622 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/5dce6406fb2742f8b4152149c4b2750a 2023-02-07 15:54:28,622 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,622 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,622 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:28,622 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:28,625 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5dce6406fb2742f8b4152149c4b2750a 2023-02-07 15:54:28,632 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/9eca8760f4a547d09c3f98a911a9909c 2023-02-07 15:54:28,636 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9eca8760f4a547d09c3f98a911a9909c 2023-02-07 15:54:28,637 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/5dce6406fb2742f8b4152149c4b2750a as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5dce6406fb2742f8b4152149c4b2750a 2023-02-07 15:54:28,642 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5dce6406fb2742f8b4152149c4b2750a 2023-02-07 15:54:28,642 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5dce6406fb2742f8b4152149c4b2750a, entries=5, sequenceid=91, filesize=5.0 K 2023-02-07 15:54:28,643 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/9eca8760f4a547d09c3f98a911a9909c as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9eca8760f4a547d09c3f98a911a9909c 2023-02-07 15:54:28,647 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9eca8760f4a547d09c3f98a911a9909c 2023-02-07 15:54:28,647 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9eca8760f4a547d09c3f98a911a9909c, entries=5, sequenceid=91, filesize=5.0 K 2023-02-07 15:54:28,648 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 33148689d0ba6f95ebbfc74509e65344 in 34ms, sequenceid=91, compaction requested=true 2023-02-07 15:54:28,648 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 33148689d0ba6f95ebbfc74509e65344: 2023-02-07 15:54:28,648 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:28,648 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:28,648 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:28,648 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,648 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:28,648 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:28,660 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,660 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:28,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:28,660 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,660 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:28,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:28,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:28,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:28,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 15:54:28,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19ebac96[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:28,662 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:28,663 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,674 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,674 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,674 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,675 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:28,675 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:28,675 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:28,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,675 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:28,675 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:28,675 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:28,675 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:28,675 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:28,674 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,675 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:28,675 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:28,685 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:28,685 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,685 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:28,695 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:28,695 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,695 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:28,706 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:28,706 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,706 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:28,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:28,706 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:28,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:28,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:28,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:28,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:28,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:28,709 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 15:54:28,710 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:28,710 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:28,710 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,710 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:28,716 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,716 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:28,716 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,716 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,716 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,717 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:28,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,717 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:28,717 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,717 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,718 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:28,719 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,719 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,719 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,751 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,751 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,751 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,751 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,751 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,751 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,751 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,751 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,751 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:28,751 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:28,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,751 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,751 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,752 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:28,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,751 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,752 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:28,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,752 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,752 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,752 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,752 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,752 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,752 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,752 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,852 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:28,852 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:28,852 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:28,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:28,860 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:28,860 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:28,861 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:54:28,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:54:28,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,862 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:28,862 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,862 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 15:54:28,863 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 15:54:28,863 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,863 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,885 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,885 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,885 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,885 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,885 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,885 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:54:28,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:28,886 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,886 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:28,886 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:28,887 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,886 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 15:54:28,887 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:28,886 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,887 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:28,887 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 15:54:28,887 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:54:28,887 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:28,887 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 15:54:28,887 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:28,887 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:54:28,887 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:54:28,887 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:28,887 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:28,887 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:28,887 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. started... 2023-02-07 15:54:28,887 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 33148689d0ba6f95ebbfc74509e65344 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-07 15:54:28,895 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,895 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/6673b8b685b9445ead193c1add8d73ac 2023-02-07 15:54:28,899 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6673b8b685b9445ead193c1add8d73ac 2023-02-07 15:54:28,906 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,906 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,906 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:28,906 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,906 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:28,906 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/0d3f48279e6d4061b7224d53a24863ec 2023-02-07 15:54:28,909 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0d3f48279e6d4061b7224d53a24863ec 2023-02-07 15:54:28,910 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/6673b8b685b9445ead193c1add8d73ac as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6673b8b685b9445ead193c1add8d73ac 2023-02-07 15:54:28,914 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6673b8b685b9445ead193c1add8d73ac 2023-02-07 15:54:28,914 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6673b8b685b9445ead193c1add8d73ac, entries=5, sequenceid=99, filesize=5.0 K 2023-02-07 15:54:28,915 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/0d3f48279e6d4061b7224d53a24863ec as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/0d3f48279e6d4061b7224d53a24863ec 2023-02-07 15:54:28,918 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0d3f48279e6d4061b7224d53a24863ec 2023-02-07 15:54:28,919 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/0d3f48279e6d4061b7224d53a24863ec, entries=5, sequenceid=99, filesize=5.0 K 2023-02-07 15:54:28,919 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 33148689d0ba6f95ebbfc74509e65344 in 32ms, sequenceid=99, compaction requested=true 2023-02-07 15:54:28,919 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 33148689d0ba6f95ebbfc74509e65344: 2023-02-07 15:54:28,919 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:28,919 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:54:28,920 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:54:28,920 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,920 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 15:54:28,920 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 15:54:28,927 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,927 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:28,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:28,927 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:28,927 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:54:28,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:28,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:28,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 15:54:28,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@458d2b96[Count = 0] remaining members to acquire global barrier 2023-02-07 15:54:28,929 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 15:54:28,929 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,937 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,937 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,937 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,938 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,938 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:54:28,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,938 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:28,938 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:28,938 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:54:28,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,938 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:28,938 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 15:54:28,938 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:28,938 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:54:28,938 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 15:54:28,938 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:54:28,948 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:28,948 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,948 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:28,958 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:28,958 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,958 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 15:54:28,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:28,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:28,958 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,958 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,958 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:28,959 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 15:54:28,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:28,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:28,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:28,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:54:28,961 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:54:28,961 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 15:54:28,961 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:54:28,961 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:54:28,961 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,961 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:54:28,969 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,969 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,969 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,969 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,969 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,969 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:54:28,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:54:28,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,969 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,970 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:54:28,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,970 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,970 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:54:28,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:54:28,971 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,972 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,972 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,979 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,979 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,979 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:54:28,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,979 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,980 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 15:54:28,980 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:54:28,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,980 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:54:28,979 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,979 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:54:28,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:54:28,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,980 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:54:28,980 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:54:28,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:54:28,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,980 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:28,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:54:28,981 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:54:28,981 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:28,981 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:28,981 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:28,981 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:29,080 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:54:29,081 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:29,081 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:29,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:54:29,084 DEBUG [Listener at localhost.localdomain/41965] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:29,085 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:29,086 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:29,086 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:29,086 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:29,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=33148689d0ba6f95ebbfc74509e65344, UNASSIGN}] 2023-02-07 15:54:29,088 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=33148689d0ba6f95ebbfc74509e65344, UNASSIGN 2023-02-07 15:54:29,088 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=33148689d0ba6f95ebbfc74509e65344, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:29,088 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675785269088"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785269088"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785269088"}]},"ts":"1675785269088"} 2023-02-07 15:54:29,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 33148689d0ba6f95ebbfc74509e65344, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:29,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 33148689d0ba6f95ebbfc74509e65344, disabling compactions & flushes 2023-02-07 15:54:29,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:29,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:29,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. after waiting 0 ms 2023-02-07 15:54:29,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:29,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 33148689d0ba6f95ebbfc74509e65344 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-07 15:54:29,251 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/63b6c91d366441e5b98b5e450cfd6102 2023-02-07 15:54:29,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63b6c91d366441e5b98b5e450cfd6102 2023-02-07 15:54:29,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/909c8a5460bb4d8b8aae1cbaf8118516 2023-02-07 15:54:29,266 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 909c8a5460bb4d8b8aae1cbaf8118516 2023-02-07 15:54:29,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf1/63b6c91d366441e5b98b5e450cfd6102 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/63b6c91d366441e5b98b5e450cfd6102 2023-02-07 15:54:29,271 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63b6c91d366441e5b98b5e450cfd6102 2023-02-07 15:54:29,271 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/63b6c91d366441e5b98b5e450cfd6102, entries=1, sequenceid=103, filesize=4.9 K 2023-02-07 15:54:29,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.tmp/cf2/909c8a5460bb4d8b8aae1cbaf8118516 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/909c8a5460bb4d8b8aae1cbaf8118516 2023-02-07 15:54:29,276 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 909c8a5460bb4d8b8aae1cbaf8118516 2023-02-07 15:54:29,277 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/909c8a5460bb4d8b8aae1cbaf8118516, entries=1, sequenceid=103, filesize=4.9 K 2023-02-07 15:54:29,277 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 33148689d0ba6f95ebbfc74509e65344 in 35ms, sequenceid=103, compaction requested=true 2023-02-07 15:54:29,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-07 15:54:29,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:29,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. 2023-02-07 15:54:29,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 33148689d0ba6f95ebbfc74509e65344: Waiting for close lock at 1675785269242Running coprocessor pre-close hooks at 1675785269242Disabling compacts and flushes for region at 1675785269242Disabling writes for close at 1675785269242Obtaining lock to block concurrent updates at 1675785269242Preparing flush snapshotting stores in 33148689d0ba6f95ebbfc74509e65344 at 1675785269242Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1675785269242Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344. at 1675785269243 (+1 ms)Flushing 33148689d0ba6f95ebbfc74509e65344/cf1: creating writer at 1675785269243Flushing 33148689d0ba6f95ebbfc74509e65344/cf1: appending metadata at 1675785269245 (+2 ms)Flushing 33148689d0ba6f95ebbfc74509e65344/cf1: closing flushed file at 1675785269245Flushing 33148689d0ba6f95ebbfc74509e65344/cf2: creating writer at 1675785269255 (+10 ms)Flushing 33148689d0ba6f95ebbfc74509e65344/cf2: appending metadata at 1675785269256 (+1 ms)Flushing 33148689d0ba6f95ebbfc74509e65344/cf2: closing flushed file at 1675785269256Flushing 33148689d0ba6f95ebbfc74509e65344/cf1: reopening flushed file at 1675785269268 (+12 ms)Flushing 33148689d0ba6f95ebbfc74509e65344/cf2: reopening flushed file at 1675785269272 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 33148689d0ba6f95ebbfc74509e65344 in 35ms, sequenceid=103, compaction requested=true at 1675785269278 (+6 ms)Writing region close event to WAL at 1675785269285 (+7 ms)Running coprocessor post-close hooks at 1675785269289 (+4 ms)Closed at 1675785269289 2023-02-07 15:54:29,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,291 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=33148689d0ba6f95ebbfc74509e65344, regionState=CLOSED 2023-02-07 15:54:29,291 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675785269291"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785269291"}]},"ts":"1675785269291"} 2023-02-07 15:54:29,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-07 15:54:29,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 33148689d0ba6f95ebbfc74509e65344, server=jenkins-hbase12.apache.org,38629,1675785236797 in 202 msec 2023-02-07 15:54:29,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-07 15:54:29,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=33148689d0ba6f95ebbfc74509e65344, UNASSIGN in 206 msec 2023-02-07 15:54:29,305 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=33148689d0ba6f95ebbfc74509e65344, threads=16 2023-02-07 15:54:29,306 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/0d3f48279e6d4061b7224d53a24863ec for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,306 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/1f9a82973f3441019b15a1729eccee63 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,306 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/55a5b4e723b54362ad08f7c6c952ac2f for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,313 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/5cf1b97a9fd14df985aeee1a531d1227 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,313 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/909c8a5460bb4d8b8aae1cbaf8118516 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,313 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,313 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9eca8760f4a547d09c3f98a911a9909c for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,314 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/c585b51e95ad4dbea65db42982d7d7e7 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,314 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/bc68958b9c3c4bc79a465a41c2db5fdf for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,315 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 15:54:29,316 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0d3f48279e6d4061b7224d53a24863ec 2023-02-07 15:54:29,317 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/d7a12805559841128b429a0dde6f1386 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,322 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/3177174266594d6cb746e113a8861a10 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,322 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/46e65271869541d3aaab446dfe308ecf for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,323 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5dce6406fb2742f8b4152149c4b2750a for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,325 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5f39c7b29bfc4ef596abb57bc6cddb61 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,332 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/63b6c91d366441e5b98b5e450cfd6102 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,334 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6673b8b685b9445ead193c1add8d73ac for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,334 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f9a82973f3441019b15a1729eccee63 2023-02-07 15:54:29,340 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/55a5b4e723b54362ad08f7c6c952ac2f, top=false 2023-02-07 15:54:29,340 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/c585b51e95ad4dbea65db42982d7d7e7, top=false 2023-02-07 15:54:29,349 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5dce6406fb2742f8b4152149c4b2750a 2023-02-07 15:54:29,350 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 909c8a5460bb4d8b8aae1cbaf8118516 2023-02-07 15:54:29,352 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0d3f48279e6d4061b7224d53a24863ec 2023-02-07 15:54:29,352 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/0d3f48279e6d4061b7224d53a24863ec, top=true 2023-02-07 15:54:29,353 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f39c7b29bfc4ef596abb57bc6cddb61 2023-02-07 15:54:29,354 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63b6c91d366441e5b98b5e450cfd6102 2023-02-07 15:54:29,354 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6673b8b685b9445ead193c1add8d73ac 2023-02-07 15:54:29,355 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9eca8760f4a547d09c3f98a911a9909c 2023-02-07 15:54:29,358 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/d7a12805559841128b429a0dde6f1386, top=false 2023-02-07 15:54:29,361 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 909c8a5460bb4d8b8aae1cbaf8118516 2023-02-07 15:54:29,361 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/909c8a5460bb4d8b8aae1cbaf8118516, top=true 2023-02-07 15:54:29,370 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5dce6406fb2742f8b4152149c4b2750a 2023-02-07 15:54:29,371 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5dce6406fb2742f8b4152149c4b2750a, top=true 2023-02-07 15:54:29,378 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9eca8760f4a547d09c3f98a911a9909c 2023-02-07 15:54:29,378 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9eca8760f4a547d09c3f98a911a9909c, top=true 2023-02-07 15:54:29,392 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.splits/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c585b51e95ad4dbea65db42982d7d7e7 for child: 8484d4fdd12a9c6089324f81f4671ae7, parent: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,395 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/c585b51e95ad4dbea65db42982d7d7e7 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,395 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6d35e862e6ae4513b97ed6281ec5e495 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,398 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6d35e862e6ae4513b97ed6281ec5e495, top=false 2023-02-07 15:54:29,409 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.splits/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-5dce6406fb2742f8b4152149c4b2750a for child: 1b6585777b9c8a7031889d672dbf4d81, parent: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,409 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5dce6406fb2742f8b4152149c4b2750a for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,409 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/862a9db3a85944f984190fba7572a5ae for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,412 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6673b8b685b9445ead193c1add8d73ac 2023-02-07 15:54:29,412 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f9a82973f3441019b15a1729eccee63 2023-02-07 15:54:29,412 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6673b8b685b9445ead193c1add8d73ac, top=true 2023-02-07 15:54:29,412 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63b6c91d366441e5b98b5e450cfd6102 2023-02-07 15:54:29,412 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/63b6c91d366441e5b98b5e450cfd6102, top=true 2023-02-07 15:54:29,417 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.splits/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-55a5b4e723b54362ad08f7c6c952ac2f for child: 8484d4fdd12a9c6089324f81f4671ae7, parent: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,420 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/55a5b4e723b54362ad08f7c6c952ac2f for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,420 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/bd94f04f7ae04647b0a1056004c41a1b for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,423 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/bd94f04f7ae04647b0a1056004c41a1b, top=false 2023-02-07 15:54:29,433 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.splits/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-d7a12805559841128b429a0dde6f1386 for child: 8484d4fdd12a9c6089324f81f4671ae7, parent: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,436 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/d7a12805559841128b429a0dde6f1386 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,436 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/c67fb23ee9dd4c79a232cde1e8b8a0e0 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,437 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.splits/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-9eca8760f4a547d09c3f98a911a9909c for child: 1b6585777b9c8a7031889d672dbf4d81, parent: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,437 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9eca8760f4a547d09c3f98a911a9909c for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,439 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/c67fb23ee9dd4c79a232cde1e8b8a0e0, top=false 2023-02-07 15:54:29,457 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.splits/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-909c8a5460bb4d8b8aae1cbaf8118516 for child: 1b6585777b9c8a7031889d672dbf4d81, parent: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,457 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.splits/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-0d3f48279e6d4061b7224d53a24863ec for child: 1b6585777b9c8a7031889d672dbf4d81, parent: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,457 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/0d3f48279e6d4061b7224d53a24863ec for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,457 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/909c8a5460bb4d8b8aae1cbaf8118516 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,457 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.splits/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6d35e862e6ae4513b97ed6281ec5e495 for child: 8484d4fdd12a9c6089324f81f4671ae7, parent: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,466 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.splits/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-63b6c91d366441e5b98b5e450cfd6102 for child: 1b6585777b9c8a7031889d672dbf4d81, parent: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,466 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/63b6c91d366441e5b98b5e450cfd6102 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,471 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f39c7b29bfc4ef596abb57bc6cddb61 2023-02-07 15:54:29,471 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6d35e862e6ae4513b97ed6281ec5e495 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,483 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,489 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.splits/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6673b8b685b9445ead193c1add8d73ac for child: 1b6585777b9c8a7031889d672dbf4d81, parent: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,489 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6673b8b685b9445ead193c1add8d73ac for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,490 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/1f9a82973f3441019b15a1729eccee63 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,501 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.splits/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-bd94f04f7ae04647b0a1056004c41a1b for child: 8484d4fdd12a9c6089324f81f4671ae7, parent: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,510 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/bd94f04f7ae04647b0a1056004c41a1b for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,516 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/46e65271869541d3aaab446dfe308ecf for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,516 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/.splits/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c67fb23ee9dd4c79a232cde1e8b8a0e0 for child: 8484d4fdd12a9c6089324f81f4671ae7, parent: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,520 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/c67fb23ee9dd4c79a232cde1e8b8a0e0 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,529 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5f39c7b29bfc4ef596abb57bc6cddb61 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,529 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/3177174266594d6cb746e113a8861a10 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,533 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/5cf1b97a9fd14df985aeee1a531d1227 for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,533 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/862a9db3a85944f984190fba7572a5ae for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,909 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/bc68958b9c3c4bc79a465a41c2db5fdf for region: 33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:29,909 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 33148689d0ba6f95ebbfc74509e65344 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 15:54:29,944 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-07 15:54:29,946 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-07 15:54:29,949 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675785269949"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1675785269949"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1675785269949"}]},"ts":"1675785269949"} 2023-02-07 15:54:29,949 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675785269949"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785269949"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785269949"}]},"ts":"1675785269949"} 2023-02-07 15:54:29,949 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675785269949"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785269949"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785269949"}]},"ts":"1675785269949"} 2023-02-07 15:54:29,953 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8484d4fdd12a9c6089324f81f4671ae7, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1b6585777b9c8a7031889d672dbf4d81, ASSIGN}] 2023-02-07 15:54:29,954 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1b6585777b9c8a7031889d672dbf4d81, ASSIGN 2023-02-07 15:54:29,955 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1b6585777b9c8a7031889d672dbf4d81, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:54:29,956 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8484d4fdd12a9c6089324f81f4671ae7, ASSIGN 2023-02-07 15:54:29,956 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8484d4fdd12a9c6089324f81f4671ae7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:54:30,105 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 15:54:30,107 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=8484d4fdd12a9c6089324f81f4671ae7, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:30,107 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675785270107"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785270107"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785270107"}]},"ts":"1675785270107"} 2023-02-07 15:54:30,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 8484d4fdd12a9c6089324f81f4671ae7, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:30,111 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=1b6585777b9c8a7031889d672dbf4d81, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:30,111 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675785270111"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785270111"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785270111"}]},"ts":"1675785270111"} 2023-02-07 15:54:30,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 1b6585777b9c8a7031889d672dbf4d81, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:30,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8484d4fdd12a9c6089324f81f4671ae7, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.', STARTKEY => '', ENDKEY => '60'} 2023-02-07 15:54:30,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:30,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 8484d4fdd12a9c6089324f81f4671ae7 2023-02-07 15:54:30,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:30,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 8484d4fdd12a9c6089324f81f4671ae7 2023-02-07 15:54:30,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 8484d4fdd12a9c6089324f81f4671ae7 2023-02-07 15:54:30,265 INFO [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8484d4fdd12a9c6089324f81f4671ae7 2023-02-07 15:54:30,266 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1 2023-02-07 15:54:30,266 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1 2023-02-07 15:54:30,266 INFO [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8484d4fdd12a9c6089324f81f4671ae7 columnFamilyName cf1 2023-02-07 15:54:30,278 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/3177174266594d6cb746e113a8861a10.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/3177174266594d6cb746e113a8861a10-bottom 2023-02-07 15:54:30,282 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/46e65271869541d3aaab446dfe308ecf.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/46e65271869541d3aaab446dfe308ecf-bottom 2023-02-07 15:54:30,286 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:30,288 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5f39c7b29bfc4ef596abb57bc6cddb61-bottom 2023-02-07 15:54:30,292 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/862a9db3a85944f984190fba7572a5ae.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/862a9db3a85944f984190fba7572a5ae-bottom 2023-02-07 15:54:30,296 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6d35e862e6ae4513b97ed6281ec5e495 2023-02-07 15:54:30,300 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-bd94f04f7ae04647b0a1056004c41a1b 2023-02-07 15:54:30,303 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c67fb23ee9dd4c79a232cde1e8b8a0e0 2023-02-07 15:54:30,304 INFO [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(310): Store=8484d4fdd12a9c6089324f81f4671ae7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:30,304 INFO [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8484d4fdd12a9c6089324f81f4671ae7 2023-02-07 15:54:30,305 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2 2023-02-07 15:54:30,305 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2 2023-02-07 15:54:30,305 INFO [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8484d4fdd12a9c6089324f81f4671ae7 columnFamilyName cf2 2023-02-07 15:54:30,316 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:30,318 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/1f9a82973f3441019b15a1729eccee63-bottom 2023-02-07 15:54:30,322 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/5cf1b97a9fd14df985aeee1a531d1227.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/5cf1b97a9fd14df985aeee1a531d1227-bottom 2023-02-07 15:54:30,327 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f-bottom 2023-02-07 15:54:30,333 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/bc68958b9c3c4bc79a465a41c2db5fdf.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/bc68958b9c3c4bc79a465a41c2db5fdf-bottom 2023-02-07 15:54:30,336 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-55a5b4e723b54362ad08f7c6c952ac2f 2023-02-07 15:54:30,340 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c585b51e95ad4dbea65db42982d7d7e7 2023-02-07 15:54:30,343 DEBUG [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-d7a12805559841128b429a0dde6f1386 2023-02-07 15:54:30,344 INFO [StoreOpener-8484d4fdd12a9c6089324f81f4671ae7-1] regionserver.HStore(310): Store=8484d4fdd12a9c6089324f81f4671ae7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:30,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7 2023-02-07 15:54:30,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7 2023-02-07 15:54:30,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:30,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 8484d4fdd12a9c6089324f81f4671ae7 2023-02-07 15:54:30,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 8484d4fdd12a9c6089324f81f4671ae7; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73334538, jitterRate=0.09276977181434631}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:30,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 8484d4fdd12a9c6089324f81f4671ae7: Running coprocessor pre-open hook at 1675785270264Writing region info on filesystem at 1675785270264Initializing all the Stores at 1675785270264Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785270264Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785270265 (+1 ms)Cleaning up temporary data from old regions at 1675785270346 (+81 ms)Cleaning up detritus from prior splits at 1675785270346Running coprocessor post-open hooks at 1675785270349 (+3 ms)Region opened successfully at 1675785270349 2023-02-07 15:54:30,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7., pid=77, masterSystemTime=1675785270260 2023-02-07 15:54:30,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:30,351 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:54:30,351 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:54:30,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:54:30,351 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,351 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,351 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1912): 8484d4fdd12a9c6089324f81f4671ae7/cf1 is initiating minor compaction (all files) 2023-02-07 15:54:30,351 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1912): 8484d4fdd12a9c6089324f81f4671ae7/cf2 is initiating minor compaction (all files) 2023-02-07 15:54:30,352 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8484d4fdd12a9c6089324f81f4671ae7/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,352 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8484d4fdd12a9c6089324f81f4671ae7/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,352 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/862a9db3a85944f984190fba7572a5ae.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/862a9db3a85944f984190fba7572a5ae-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/3177174266594d6cb746e113a8861a10.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/3177174266594d6cb746e113a8861a10-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/46e65271869541d3aaab446dfe308ecf.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/46e65271869541d3aaab446dfe308ecf-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6d35e862e6ae4513b97ed6281ec5e495, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c67fb23ee9dd4c79a232cde1e8b8a0e0, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-bd94f04f7ae04647b0a1056004c41a1b, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5f39c7b29bfc4ef596abb57bc6cddb61-bottom] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/.tmp, totalSize=35.0 K 2023-02-07 15:54:30,352 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/5cf1b97a9fd14df985aeee1a531d1227.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/5cf1b97a9fd14df985aeee1a531d1227-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/bc68958b9c3c4bc79a465a41c2db5fdf.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/bc68958b9c3c4bc79a465a41c2db5fdf-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-55a5b4e723b54362ad08f7c6c952ac2f, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c585b51e95ad4dbea65db42982d7d7e7, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-d7a12805559841128b429a0dde6f1386, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/1f9a82973f3441019b15a1729eccee63-bottom] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/.tmp, totalSize=35.0 K 2023-02-07 15:54:30,352 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 862a9db3a85944f984190fba7572a5ae.33148689d0ba6f95ebbfc74509e65344, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785266285 2023-02-07 15:54:30,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,353 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 9e8fb9ee754b4cc6bf2707fcfa6d7a2f.33148689d0ba6f95ebbfc74509e65344, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785266285 2023-02-07 15:54:30,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,353 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 3177174266594d6cb746e113a8861a10.33148689d0ba6f95ebbfc74509e65344, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785266550 2023-02-07 15:54:30,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1b6585777b9c8a7031889d672dbf4d81, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.', STARTKEY => '60', ENDKEY => ''} 2023-02-07 15:54:30,353 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=8484d4fdd12a9c6089324f81f4671ae7, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:30,353 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 46e65271869541d3aaab446dfe308ecf.33148689d0ba6f95ebbfc74509e65344, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785266809 2023-02-07 15:54:30,354 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675785270353"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785270353"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785270353"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785270353"}]},"ts":"1675785270353"} 2023-02-07 15:54:30,353 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 5cf1b97a9fd14df985aeee1a531d1227.33148689d0ba6f95ebbfc74509e65344, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785266550 2023-02-07 15:54:30,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:30,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:30,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:30,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:30,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:30,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-07 15:54:30,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 8484d4fdd12a9c6089324f81f4671ae7, server=jenkins-hbase12.apache.org,38629,1675785236797 in 246 msec 2023-02-07 15:54:30,357 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6d35e862e6ae4513b97ed6281ec5e495, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785267503 2023-02-07 15:54:30,358 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8484d4fdd12a9c6089324f81f4671ae7, ASSIGN in 403 msec 2023-02-07 15:54:30,358 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c67fb23ee9dd4c79a232cde1e8b8a0e0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785267778 2023-02-07 15:54:30,358 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting bc68958b9c3c4bc79a465a41c2db5fdf.33148689d0ba6f95ebbfc74509e65344, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785266809 2023-02-07 15:54:30,358 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-bd94f04f7ae04647b0a1056004c41a1b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785268083 2023-02-07 15:54:30,358 INFO [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:30,358 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-55a5b4e723b54362ad08f7c6c952ac2f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785267503 2023-02-07 15:54:30,359 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675785268314 2023-02-07 15:54:30,359 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c585b51e95ad4dbea65db42982d7d7e7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785267778 2023-02-07 15:54:30,359 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-d7a12805559841128b429a0dde6f1386, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785268083 2023-02-07 15:54:30,359 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675785268314 2023-02-07 15:54:30,360 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1 2023-02-07 15:54:30,361 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1 2023-02-07 15:54:30,361 INFO [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b6585777b9c8a7031889d672dbf4d81 columnFamilyName cf1 2023-02-07 15:54:30,378 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8484d4fdd12a9c6089324f81f4671ae7#cf1#compaction#72 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:30,381 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/3177174266594d6cb746e113a8861a10.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/3177174266594d6cb746e113a8861a10-top 2023-02-07 15:54:30,387 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/46e65271869541d3aaab446dfe308ecf.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/46e65271869541d3aaab446dfe308ecf-top 2023-02-07 15:54:30,390 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:30,390 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5f39c7b29bfc4ef596abb57bc6cddb61-top 2023-02-07 15:54:30,398 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] throttle.PressureAwareThroughputController(145): 8484d4fdd12a9c6089324f81f4671ae7#cf2#compaction#73 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:30,398 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/862a9db3a85944f984190fba7572a5ae.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/862a9db3a85944f984190fba7572a5ae-top 2023-02-07 15:54:30,402 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5dce6406fb2742f8b4152149c4b2750a 2023-02-07 15:54:30,402 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-5dce6406fb2742f8b4152149c4b2750a 2023-02-07 15:54:30,404 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/.tmp/cf1/4c149b8c8b224088bf5ec0eae3bb6698 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/4c149b8c8b224088bf5ec0eae3bb6698 2023-02-07 15:54:30,405 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63b6c91d366441e5b98b5e450cfd6102 2023-02-07 15:54:30,405 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-63b6c91d366441e5b98b5e450cfd6102 2023-02-07 15:54:30,408 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6673b8b685b9445ead193c1add8d73ac 2023-02-07 15:54:30,409 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6673b8b685b9445ead193c1add8d73ac 2023-02-07 15:54:30,409 INFO [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(310): Store=1b6585777b9c8a7031889d672dbf4d81/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:30,409 INFO [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:30,409 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8484d4fdd12a9c6089324f81f4671ae7/cf1 of 8484d4fdd12a9c6089324f81f4671ae7 into 4c149b8c8b224088bf5ec0eae3bb6698(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:30,410 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8484d4fdd12a9c6089324f81f4671ae7: 2023-02-07 15:54:30,410 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7., storeName=8484d4fdd12a9c6089324f81f4671ae7/cf1, priority=9, startTime=1675785270350; duration=0sec 2023-02-07 15:54:30,410 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:30,410 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2 2023-02-07 15:54:30,410 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2 2023-02-07 15:54:30,410 INFO [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b6585777b9c8a7031889d672dbf4d81 columnFamilyName cf2 2023-02-07 15:54:30,420 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/.tmp/cf2/fad449c6f8b04b6188dd0d9acb194805 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/fad449c6f8b04b6188dd0d9acb194805 2023-02-07 15:54:30,422 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:30,423 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/1f9a82973f3441019b15a1729eccee63-top 2023-02-07 15:54:30,425 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8484d4fdd12a9c6089324f81f4671ae7/cf2 of 8484d4fdd12a9c6089324f81f4671ae7 into fad449c6f8b04b6188dd0d9acb194805(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:30,425 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8484d4fdd12a9c6089324f81f4671ae7: 2023-02-07 15:54:30,425 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7., storeName=8484d4fdd12a9c6089324f81f4671ae7/cf2, priority=9, startTime=1675785270351; duration=0sec 2023-02-07 15:54:30,425 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:30,429 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/5cf1b97a9fd14df985aeee1a531d1227.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/5cf1b97a9fd14df985aeee1a531d1227-top 2023-02-07 15:54:30,433 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f-top 2023-02-07 15:54:30,437 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/bc68958b9c3c4bc79a465a41c2db5fdf.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/bc68958b9c3c4bc79a465a41c2db5fdf-top 2023-02-07 15:54:30,445 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0d3f48279e6d4061b7224d53a24863ec 2023-02-07 15:54:30,445 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-0d3f48279e6d4061b7224d53a24863ec 2023-02-07 15:54:30,449 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 909c8a5460bb4d8b8aae1cbaf8118516 2023-02-07 15:54:30,449 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-909c8a5460bb4d8b8aae1cbaf8118516 2023-02-07 15:54:30,454 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9eca8760f4a547d09c3f98a911a9909c 2023-02-07 15:54:30,454 DEBUG [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-9eca8760f4a547d09c3f98a911a9909c 2023-02-07 15:54:30,454 INFO [StoreOpener-1b6585777b9c8a7031889d672dbf4d81-1] regionserver.HStore(310): Store=1b6585777b9c8a7031889d672dbf4d81/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:30,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:30,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:30,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:54:30,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:30,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 1b6585777b9c8a7031889d672dbf4d81; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73460152, jitterRate=0.09464156627655029}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:54:30,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 1b6585777b9c8a7031889d672dbf4d81: Running coprocessor pre-open hook at 1675785270355Writing region info on filesystem at 1675785270355Initializing all the Stores at 1675785270358 (+3 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785270358Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785270358Cleaning up temporary data from old regions at 1675785270456 (+98 ms)Cleaning up detritus from prior splits at 1675785270457 (+1 ms)Running coprocessor post-open hooks at 1675785270459 (+2 ms)Region opened successfully at 1675785270459 2023-02-07 15:54:30,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81., pid=78, masterSystemTime=1675785270260 2023-02-07 15:54:30,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:30,461 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:54:30,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:54:30,461 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:54:30,462 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,462 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1912): 1b6585777b9c8a7031889d672dbf4d81/cf1 is initiating minor compaction (all files) 2023-02-07 15:54:30,462 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,462 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1912): 1b6585777b9c8a7031889d672dbf4d81/cf2 is initiating minor compaction (all files) 2023-02-07 15:54:30,462 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1b6585777b9c8a7031889d672dbf4d81/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,462 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1b6585777b9c8a7031889d672dbf4d81/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,462 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/862a9db3a85944f984190fba7572a5ae.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/862a9db3a85944f984190fba7572a5ae-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/3177174266594d6cb746e113a8861a10.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/3177174266594d6cb746e113a8861a10-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/46e65271869541d3aaab446dfe308ecf.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/46e65271869541d3aaab446dfe308ecf-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5f39c7b29bfc4ef596abb57bc6cddb61-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-5dce6406fb2742f8b4152149c4b2750a, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6673b8b685b9445ead193c1add8d73ac, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-63b6c91d366441e5b98b5e450cfd6102] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/.tmp, totalSize=34.8 K 2023-02-07 15:54:30,462 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/5cf1b97a9fd14df985aeee1a531d1227.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/5cf1b97a9fd14df985aeee1a531d1227-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/bc68958b9c3c4bc79a465a41c2db5fdf.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/bc68958b9c3c4bc79a465a41c2db5fdf-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/1f9a82973f3441019b15a1729eccee63-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-9eca8760f4a547d09c3f98a911a9909c, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-0d3f48279e6d4061b7224d53a24863ec, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-909c8a5460bb4d8b8aae1cbaf8118516] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/.tmp, totalSize=34.8 K 2023-02-07 15:54:30,463 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 862a9db3a85944f984190fba7572a5ae.33148689d0ba6f95ebbfc74509e65344, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675785266285 2023-02-07 15:54:30,463 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 9e8fb9ee754b4cc6bf2707fcfa6d7a2f.33148689d0ba6f95ebbfc74509e65344, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675785266285 2023-02-07 15:54:30,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,464 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=1b6585777b9c8a7031889d672dbf4d81, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:30,464 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 3177174266594d6cb746e113a8861a10.33148689d0ba6f95ebbfc74509e65344, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675785266550 2023-02-07 15:54:30,464 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 5cf1b97a9fd14df985aeee1a531d1227.33148689d0ba6f95ebbfc74509e65344, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675785266550 2023-02-07 15:54:30,464 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675785270464"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785270464"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785270464"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785270464"}]},"ts":"1675785270464"} 2023-02-07 15:54:30,465 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 46e65271869541d3aaab446dfe308ecf.33148689d0ba6f95ebbfc74509e65344, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675785266809 2023-02-07 15:54:30,465 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675785268314 2023-02-07 15:54:30,465 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting bc68958b9c3c4bc79a465a41c2db5fdf.33148689d0ba6f95ebbfc74509e65344, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675785266809 2023-02-07 15:54:30,465 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-5dce6406fb2742f8b4152149c4b2750a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-07 15:54:30,465 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675785268314 2023-02-07 15:54:30,466 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6673b8b685b9445ead193c1add8d73ac, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-07 15:54:30,466 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-9eca8760f4a547d09c3f98a911a9909c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-07 15:54:30,466 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-63b6c91d366441e5b98b5e450cfd6102, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-07 15:54:30,467 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-0d3f48279e6d4061b7224d53a24863ec, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-07 15:54:30,467 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-909c8a5460bb4d8b8aae1cbaf8118516, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-07 15:54:30,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-07 15:54:30,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 1b6585777b9c8a7031889d672dbf4d81, server=jenkins-hbase12.apache.org,38629,1675785236797 in 354 msec 2023-02-07 15:54:30,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-07 15:54:30,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1b6585777b9c8a7031889d672dbf4d81, ASSIGN in 517 msec 2023-02-07 15:54:30,475 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 in 1.3890 sec 2023-02-07 15:54:30,487 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1b6585777b9c8a7031889d672dbf4d81#cf1#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:30,488 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] throttle.PressureAwareThroughputController(145): 1b6585777b9c8a7031889d672dbf4d81#cf2#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:30,512 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for f5305204b34f4e88a5e1a660c166a10d: NONE, but ROW specified in column family configuration 2023-02-07 15:54:30,512 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 5a91ec9778234c0bb02479e45875ab9b: NONE, but ROW specified in column family configuration 2023-02-07 15:54:30,513 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/.tmp/cf1/5a91ec9778234c0bb02479e45875ab9b as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/5a91ec9778234c0bb02479e45875ab9b 2023-02-07 15:54:30,513 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/.tmp/cf2/f5305204b34f4e88a5e1a660c166a10d as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/f5305204b34f4e88a5e1a660c166a10d 2023-02-07 15:54:30,518 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for f5305204b34f4e88a5e1a660c166a10d: NONE, but ROW specified in column family configuration 2023-02-07 15:54:30,518 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 5a91ec9778234c0bb02479e45875ab9b: NONE, but ROW specified in column family configuration 2023-02-07 15:54:30,519 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1b6585777b9c8a7031889d672dbf4d81/cf1 of 1b6585777b9c8a7031889d672dbf4d81 into 5a91ec9778234c0bb02479e45875ab9b(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:30,519 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1b6585777b9c8a7031889d672dbf4d81/cf2 of 1b6585777b9c8a7031889d672dbf4d81 into f5305204b34f4e88a5e1a660c166a10d(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:30,519 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1b6585777b9c8a7031889d672dbf4d81: 2023-02-07 15:54:30,519 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1b6585777b9c8a7031889d672dbf4d81: 2023-02-07 15:54:30,519 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81., storeName=1b6585777b9c8a7031889d672dbf4d81/cf1, priority=9, startTime=1675785270460; duration=0sec 2023-02-07 15:54:30,519 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81., storeName=1b6585777b9c8a7031889d672dbf4d81/cf2, priority=9, startTime=1675785270461; duration=0sec 2023-02-07 15:54:30,519 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:30,519 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:30,597 DEBUG [Listener at localhost.localdomain/41965] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-07 15:54:30,601 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:49166, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-07 15:54:30,601 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,602 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-07 15:54:30,602 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.HStore(1912): 8484d4fdd12a9c6089324f81f4671ae7/cf1 is initiating major compaction (all files) 2023-02-07 15:54:30,602 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4cccad94; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:30,603 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8484d4fdd12a9c6089324f81f4671ae7/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,603 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-07 15:54:30,603 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/4c149b8c8b224088bf5ec0eae3bb6698] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/.tmp, totalSize=6.8 K 2023-02-07 15:54:30,603 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.HStore(1912): 8484d4fdd12a9c6089324f81f4671ae7/cf2 is initiating major compaction (all files) 2023-02-07 15:54:30,603 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3c6b0ded; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:54:30,603 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8484d4fdd12a9c6089324f81f4671ae7/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,603 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/fad449c6f8b04b6188dd0d9acb194805] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/.tmp, totalSize=6.8 K 2023-02-07 15:54:30,603 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 4c149b8c8b224088bf5ec0eae3bb6698, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675785266285 2023-02-07 15:54:30,603 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting fad449c6f8b04b6188dd0d9acb194805, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675785266285 2023-02-07 15:54:30,605 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38629] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,605 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38629] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-07 15:54:30,606 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38629] regionserver.HStore(1912): 1b6585777b9c8a7031889d672dbf4d81/cf1 is initiating major compaction (all files) 2023-02-07 15:54:30,606 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38629] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@39e0addc; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:54:30,606 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38629] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-07 15:54:30,606 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38629] regionserver.HStore(1912): 1b6585777b9c8a7031889d672dbf4d81/cf2 is initiating major compaction (all files) 2023-02-07 15:54:30,606 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38629] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@33bb55a6; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-07 15:54:30,607 INFO [Listener at localhost.localdomain/41965] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-07 15:54:30,607 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] throttle.PressureAwareThroughputController(145): 8484d4fdd12a9c6089324f81f4671ae7#cf2#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:30,607 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8484d4fdd12a9c6089324f81f4671ae7#cf1#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:30,621 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. is not mergeable because it has references 2023-02-07 15:54:30,622 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/.tmp/cf1/18cbb9816e5546eda64c27c13c1dba0f as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/18cbb9816e5546eda64c27c13c1dba0f 2023-02-07 15:54:30,622 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38629] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. is not mergeable because it has references 2023-02-07 15:54:30,623 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/.tmp/cf2/98b0b3e49e454cd8882ac89e13ad2265 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/98b0b3e49e454cd8882ac89e13ad2265 2023-02-07 15:54:30,627 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 8484d4fdd12a9c6089324f81f4671ae7/cf1 of 8484d4fdd12a9c6089324f81f4671ae7 into 18cbb9816e5546eda64c27c13c1dba0f(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:30,627 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8484d4fdd12a9c6089324f81f4671ae7: 2023-02-07 15:54:30,630 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675785270602; duration=0sec 2023-02-07 15:54:30,630 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-07 15:54:30,630 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1b6585777b9c8a7031889d672dbf4d81/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,630 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/5a91ec9778234c0bb02479e45875ab9b] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/.tmp, totalSize=5.0 K 2023-02-07 15:54:30,630 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 8484d4fdd12a9c6089324f81f4671ae7/cf2 of 8484d4fdd12a9c6089324f81f4671ae7 into 98b0b3e49e454cd8882ac89e13ad2265(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:30,630 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8484d4fdd12a9c6089324f81f4671ae7: 2023-02-07 15:54:30,630 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675785270603; duration=0sec 2023-02-07 15:54:30,630 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 5a91ec9778234c0bb02479e45875ab9b, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-07 15:54:30,631 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:54:30,631 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1b6585777b9c8a7031889d672dbf4d81/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,631 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/f5305204b34f4e88a5e1a660c166a10d] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/.tmp, totalSize=5.0 K 2023-02-07 15:54:30,631 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting f5305204b34f4e88a5e1a660c166a10d, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-07 15:54:30,633 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1b6585777b9c8a7031889d672dbf4d81#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:30,633 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] throttle.PressureAwareThroughputController(145): 1b6585777b9c8a7031889d672dbf4d81#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:54:30,650 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 9f2ccf9e98fc4297a28ef002419ccf3a: NONE, but ROW specified in column family configuration 2023-02-07 15:54:30,650 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 7e802cf10f374f319db6d4559c02d18f: NONE, but ROW specified in column family configuration 2023-02-07 15:54:30,651 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/.tmp/cf1/7e802cf10f374f319db6d4559c02d18f as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/7e802cf10f374f319db6d4559c02d18f 2023-02-07 15:54:30,651 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/.tmp/cf2/9f2ccf9e98fc4297a28ef002419ccf3a as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/9f2ccf9e98fc4297a28ef002419ccf3a 2023-02-07 15:54:30,654 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 7e802cf10f374f319db6d4559c02d18f: NONE, but ROW specified in column family configuration 2023-02-07 15:54:30,654 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 9f2ccf9e98fc4297a28ef002419ccf3a: NONE, but ROW specified in column family configuration 2023-02-07 15:54:30,655 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 1b6585777b9c8a7031889d672dbf4d81/cf1 of 1b6585777b9c8a7031889d672dbf4d81 into 7e802cf10f374f319db6d4559c02d18f(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:30,655 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 1b6585777b9c8a7031889d672dbf4d81/cf2 of 1b6585777b9c8a7031889d672dbf4d81 into 9f2ccf9e98fc4297a28ef002419ccf3a(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:54:30,655 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1b6585777b9c8a7031889d672dbf4d81: 2023-02-07 15:54:30,655 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1b6585777b9c8a7031889d672dbf4d81: 2023-02-07 15:54:30,655 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675785270605; duration=0sec 2023-02-07 15:54:30,655 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675785270606; duration=0sec 2023-02-07 15:54:30,655 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:30,655 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:54:30,731 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. is not mergeable because it has references 2023-02-07 15:54:30,732 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38629] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. is not mergeable because it has references 2023-02-07 15:54:30,738 DEBUG [Listener at localhost.localdomain/41965] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-07 15:54:30,740 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:30,740 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:30,740 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:30,741 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:30,741 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:30,741 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:30,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:30,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:30,744 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:30,744 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:30,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-07 15:54:30,749 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785270749"}]},"ts":"1675785270749"} 2023-02-07 15:54:30,750 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-07 15:54:30,818 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-07 15:54:30,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8484d4fdd12a9c6089324f81f4671ae7, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1b6585777b9c8a7031889d672dbf4d81, UNASSIGN}] 2023-02-07 15:54:30,819 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1b6585777b9c8a7031889d672dbf4d81, UNASSIGN 2023-02-07 15:54:30,820 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=1b6585777b9c8a7031889d672dbf4d81, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:30,820 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675785270820"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785270820"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785270820"}]},"ts":"1675785270820"} 2023-02-07 15:54:30,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 1b6585777b9c8a7031889d672dbf4d81, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:30,823 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8484d4fdd12a9c6089324f81f4671ae7, UNASSIGN 2023-02-07 15:54:30,823 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=8484d4fdd12a9c6089324f81f4671ae7, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:54:30,824 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675785270823"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785270823"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785270823"}]},"ts":"1675785270823"} 2023-02-07 15:54:30,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 8484d4fdd12a9c6089324f81f4671ae7, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:54:30,845 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:30,845 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:30,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-07 15:54:30,973 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:30,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 1b6585777b9c8a7031889d672dbf4d81, disabling compactions & flushes 2023-02-07 15:54:30,973 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. after waiting 0 ms 2023-02-07 15:54:30,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,974 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/862a9db3a85944f984190fba7572a5ae.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/862a9db3a85944f984190fba7572a5ae-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/3177174266594d6cb746e113a8861a10.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/3177174266594d6cb746e113a8861a10-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/46e65271869541d3aaab446dfe308ecf.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/46e65271869541d3aaab446dfe308ecf-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5f39c7b29bfc4ef596abb57bc6cddb61-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-5dce6406fb2742f8b4152149c4b2750a, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6673b8b685b9445ead193c1add8d73ac, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/5a91ec9778234c0bb02479e45875ab9b, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-63b6c91d366441e5b98b5e450cfd6102] to archive 2023-02-07 15:54:30,974 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:30,976 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/862a9db3a85944f984190fba7572a5ae.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/862a9db3a85944f984190fba7572a5ae.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:30,977 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/3177174266594d6cb746e113a8861a10.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/3177174266594d6cb746e113a8861a10.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:30,979 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/46e65271869541d3aaab446dfe308ecf.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/46e65271869541d3aaab446dfe308ecf.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:30,980 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:30,981 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-5dce6406fb2742f8b4152149c4b2750a to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-5dce6406fb2742f8b4152149c4b2750a 2023-02-07 15:54:30,982 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6673b8b685b9445ead193c1add8d73ac to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6673b8b685b9445ead193c1add8d73ac 2023-02-07 15:54:30,983 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/5a91ec9778234c0bb02479e45875ab9b to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/5a91ec9778234c0bb02479e45875ab9b 2023-02-07 15:54:30,983 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-63b6c91d366441e5b98b5e450cfd6102 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-63b6c91d366441e5b98b5e450cfd6102 2023-02-07 15:54:30,985 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/5cf1b97a9fd14df985aeee1a531d1227.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/5cf1b97a9fd14df985aeee1a531d1227-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/bc68958b9c3c4bc79a465a41c2db5fdf.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/bc68958b9c3c4bc79a465a41c2db5fdf-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/1f9a82973f3441019b15a1729eccee63-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-9eca8760f4a547d09c3f98a911a9909c, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-0d3f48279e6d4061b7224d53a24863ec, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/f5305204b34f4e88a5e1a660c166a10d, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-909c8a5460bb4d8b8aae1cbaf8118516] to archive 2023-02-07 15:54:30,985 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:30,987 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:30,987 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/5cf1b97a9fd14df985aeee1a531d1227.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/5cf1b97a9fd14df985aeee1a531d1227.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:30,988 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/bc68958b9c3c4bc79a465a41c2db5fdf.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/bc68958b9c3c4bc79a465a41c2db5fdf.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:30,989 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:30,990 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-9eca8760f4a547d09c3f98a911a9909c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-9eca8760f4a547d09c3f98a911a9909c 2023-02-07 15:54:30,991 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-0d3f48279e6d4061b7224d53a24863ec to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-0d3f48279e6d4061b7224d53a24863ec 2023-02-07 15:54:30,991 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/f5305204b34f4e88a5e1a660c166a10d to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/f5305204b34f4e88a5e1a660c166a10d 2023-02-07 15:54:30,992 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-909c8a5460bb4d8b8aae1cbaf8118516 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-909c8a5460bb4d8b8aae1cbaf8118516 2023-02-07 15:54:30,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-07 15:54:30,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:30,996 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81. 2023-02-07 15:54:30,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 1b6585777b9c8a7031889d672dbf4d81: Waiting for close lock at 1675785270973Running coprocessor pre-close hooks at 1675785270973Disabling compacts and flushes for region at 1675785270973Disabling writes for close at 1675785270973Writing region close event to WAL at 1675785270993 (+20 ms)Running coprocessor post-close hooks at 1675785270996 (+3 ms)Closed at 1675785270996 2023-02-07 15:54:30,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:30,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 8484d4fdd12a9c6089324f81f4671ae7 2023-02-07 15:54:30,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 8484d4fdd12a9c6089324f81f4671ae7, disabling compactions & flushes 2023-02-07 15:54:30,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. after waiting 0 ms 2023-02-07 15:54:30,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:30,997 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=1b6585777b9c8a7031889d672dbf4d81, regionState=CLOSED 2023-02-07 15:54:30,997 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675785270997"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785270997"}]},"ts":"1675785270997"} 2023-02-07 15:54:30,999 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/862a9db3a85944f984190fba7572a5ae.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/862a9db3a85944f984190fba7572a5ae-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/3177174266594d6cb746e113a8861a10.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/3177174266594d6cb746e113a8861a10-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/46e65271869541d3aaab446dfe308ecf.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/46e65271869541d3aaab446dfe308ecf-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6d35e862e6ae4513b97ed6281ec5e495, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c67fb23ee9dd4c79a232cde1e8b8a0e0, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-bd94f04f7ae04647b0a1056004c41a1b, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/4c149b8c8b224088bf5ec0eae3bb6698, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5f39c7b29bfc4ef596abb57bc6cddb61-bottom] to archive 2023-02-07 15:54:30,999 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:31,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-07 15:54:31,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 1b6585777b9c8a7031889d672dbf4d81, server=jenkins-hbase12.apache.org,38629,1675785236797 in 178 msec 2023-02-07 15:54:31,001 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/862a9db3a85944f984190fba7572a5ae.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/862a9db3a85944f984190fba7572a5ae.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:31,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1b6585777b9c8a7031889d672dbf4d81, UNASSIGN in 181 msec 2023-02-07 15:54:31,002 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/3177174266594d6cb746e113a8861a10.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/3177174266594d6cb746e113a8861a10.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:31,002 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/46e65271869541d3aaab446dfe308ecf.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/46e65271869541d3aaab446dfe308ecf.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:31,003 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6d35e862e6ae4513b97ed6281ec5e495 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-6d35e862e6ae4513b97ed6281ec5e495 2023-02-07 15:54:31,004 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c67fb23ee9dd4c79a232cde1e8b8a0e0 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c67fb23ee9dd4c79a232cde1e8b8a0e0 2023-02-07 15:54:31,005 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-bd94f04f7ae04647b0a1056004c41a1b to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-bd94f04f7ae04647b0a1056004c41a1b 2023-02-07 15:54:31,006 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/4c149b8c8b224088bf5ec0eae3bb6698 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/4c149b8c8b224088bf5ec0eae3bb6698 2023-02-07 15:54:31,006 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/5f39c7b29bfc4ef596abb57bc6cddb61.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:31,008 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/5cf1b97a9fd14df985aeee1a531d1227.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/5cf1b97a9fd14df985aeee1a531d1227-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/bc68958b9c3c4bc79a465a41c2db5fdf.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/bc68958b9c3c4bc79a465a41c2db5fdf-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-55a5b4e723b54362ad08f7c6c952ac2f, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c585b51e95ad4dbea65db42982d7d7e7, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-d7a12805559841128b429a0dde6f1386, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/fad449c6f8b04b6188dd0d9acb194805, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/1f9a82973f3441019b15a1729eccee63-bottom] to archive 2023-02-07 15:54:31,009 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:54:31,010 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:31,011 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/5cf1b97a9fd14df985aeee1a531d1227.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/5cf1b97a9fd14df985aeee1a531d1227.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:31,012 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/bc68958b9c3c4bc79a465a41c2db5fdf.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/bc68958b9c3c4bc79a465a41c2db5fdf.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:31,012 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-55a5b4e723b54362ad08f7c6c952ac2f to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-55a5b4e723b54362ad08f7c6c952ac2f 2023-02-07 15:54:31,013 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c585b51e95ad4dbea65db42982d7d7e7 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-c585b51e95ad4dbea65db42982d7d7e7 2023-02-07 15:54:31,014 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-d7a12805559841128b429a0dde6f1386 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/testSplitTableRegionDeletedRowsDaughter=33148689d0ba6f95ebbfc74509e65344-d7a12805559841128b429a0dde6f1386 2023-02-07 15:54:31,015 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/fad449c6f8b04b6188dd0d9acb194805 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/fad449c6f8b04b6188dd0d9acb194805 2023-02-07 15:54:31,016 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/1f9a82973f3441019b15a1729eccee63.33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:31,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-07 15:54:31,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:31,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7. 2023-02-07 15:54:31,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 8484d4fdd12a9c6089324f81f4671ae7: Waiting for close lock at 1675785270997Running coprocessor pre-close hooks at 1675785270997Disabling compacts and flushes for region at 1675785270997Disabling writes for close at 1675785270997Writing region close event to WAL at 1675785271017 (+20 ms)Running coprocessor post-close hooks at 1675785271019 (+2 ms)Closed at 1675785271020 (+1 ms) 2023-02-07 15:54:31,021 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 8484d4fdd12a9c6089324f81f4671ae7 2023-02-07 15:54:31,021 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=8484d4fdd12a9c6089324f81f4671ae7, regionState=CLOSED 2023-02-07 15:54:31,021 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675785271021"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785271021"}]},"ts":"1675785271021"} 2023-02-07 15:54:31,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-07 15:54:31,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 8484d4fdd12a9c6089324f81f4671ae7, server=jenkins-hbase12.apache.org,38629,1675785236797 in 197 msec 2023-02-07 15:54:31,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-07 15:54:31,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8484d4fdd12a9c6089324f81f4671ae7, UNASSIGN in 205 msec 2023-02-07 15:54:31,026 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785271026"}]},"ts":"1675785271026"} 2023-02-07 15:54:31,027 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-07 15:54:31,039 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-07 15:54:31,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 298 msec 2023-02-07 15:54:31,046 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:31,047 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:31,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-07 15:54:31,048 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-07 15:54:31,048 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:31,048 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:31,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:31,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:31,051 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:31,051 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:31,051 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:31,051 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:31,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-07 15:54:31,055 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:31,056 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:31,056 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7 2023-02-07 15:54:31,057 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/recovered.edits] 2023-02-07 15:54:31,057 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/recovered.edits] 2023-02-07 15:54:31,057 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/recovered.edits] 2023-02-07 15:54:31,060 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/7e802cf10f374f319db6d4559c02d18f to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf1/7e802cf10f374f319db6d4559c02d18f 2023-02-07 15:54:31,060 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/3177174266594d6cb746e113a8861a10 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/3177174266594d6cb746e113a8861a10 2023-02-07 15:54:31,060 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/18cbb9816e5546eda64c27c13c1dba0f to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf1/18cbb9816e5546eda64c27c13c1dba0f 2023-02-07 15:54:31,061 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/46e65271869541d3aaab446dfe308ecf to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/46e65271869541d3aaab446dfe308ecf 2023-02-07 15:54:31,062 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/9f2ccf9e98fc4297a28ef002419ccf3a to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/cf2/9f2ccf9e98fc4297a28ef002419ccf3a 2023-02-07 15:54:31,062 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/98b0b3e49e454cd8882ac89e13ad2265 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/cf2/98b0b3e49e454cd8882ac89e13ad2265 2023-02-07 15:54:31,062 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5dce6406fb2742f8b4152149c4b2750a to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5dce6406fb2742f8b4152149c4b2750a 2023-02-07 15:54:31,063 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5f39c7b29bfc4ef596abb57bc6cddb61 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/5f39c7b29bfc4ef596abb57bc6cddb61 2023-02-07 15:54:31,064 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/recovered.edits/113.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81/recovered.edits/113.seqid 2023-02-07 15:54:31,064 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/recovered.edits/113.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7/recovered.edits/113.seqid 2023-02-07 15:54:31,064 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/63b6c91d366441e5b98b5e450cfd6102 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/63b6c91d366441e5b98b5e450cfd6102 2023-02-07 15:54:31,065 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:54:31,065 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8484d4fdd12a9c6089324f81f4671ae7 2023-02-07 15:54:31,066 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6673b8b685b9445ead193c1add8d73ac to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6673b8b685b9445ead193c1add8d73ac 2023-02-07 15:54:31,067 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6d35e862e6ae4513b97ed6281ec5e495 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/6d35e862e6ae4513b97ed6281ec5e495 2023-02-07 15:54:31,069 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/862a9db3a85944f984190fba7572a5ae to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/862a9db3a85944f984190fba7572a5ae 2023-02-07 15:54:31,070 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/bd94f04f7ae04647b0a1056004c41a1b to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/bd94f04f7ae04647b0a1056004c41a1b 2023-02-07 15:54:31,071 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/c67fb23ee9dd4c79a232cde1e8b8a0e0 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf1/c67fb23ee9dd4c79a232cde1e8b8a0e0 2023-02-07 15:54:31,074 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/0d3f48279e6d4061b7224d53a24863ec to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/0d3f48279e6d4061b7224d53a24863ec 2023-02-07 15:54:31,075 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/1f9a82973f3441019b15a1729eccee63 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/1f9a82973f3441019b15a1729eccee63 2023-02-07 15:54:31,077 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/55a5b4e723b54362ad08f7c6c952ac2f to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/55a5b4e723b54362ad08f7c6c952ac2f 2023-02-07 15:54:31,078 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/5cf1b97a9fd14df985aeee1a531d1227 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/5cf1b97a9fd14df985aeee1a531d1227 2023-02-07 15:54:31,080 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/909c8a5460bb4d8b8aae1cbaf8118516 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/909c8a5460bb4d8b8aae1cbaf8118516 2023-02-07 15:54:31,081 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9e8fb9ee754b4cc6bf2707fcfa6d7a2f 2023-02-07 15:54:31,082 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9eca8760f4a547d09c3f98a911a9909c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/9eca8760f4a547d09c3f98a911a9909c 2023-02-07 15:54:31,084 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/bc68958b9c3c4bc79a465a41c2db5fdf to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/bc68958b9c3c4bc79a465a41c2db5fdf 2023-02-07 15:54:31,085 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/c585b51e95ad4dbea65db42982d7d7e7 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/c585b51e95ad4dbea65db42982d7d7e7 2023-02-07 15:54:31,086 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/d7a12805559841128b429a0dde6f1386 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/cf2/d7a12805559841128b429a0dde6f1386 2023-02-07 15:54:31,089 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/recovered.edits/106.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344/recovered.edits/106.seqid 2023-02-07 15:54:31,089 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/33148689d0ba6f95ebbfc74509e65344 2023-02-07 15:54:31,089 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-07 15:54:31,091 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:31,094 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-07 15:54:31,096 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-07 15:54:31,098 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:31,098 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-07 15:54:31,098 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785271098"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:31,098 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785271098"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:31,098 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785271098"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:31,100 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 15:54:31,100 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 33148689d0ba6f95ebbfc74509e65344, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675785265780.33148689d0ba6f95ebbfc74509e65344.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 8484d4fdd12a9c6089324f81f4671ae7, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675785269084.8484d4fdd12a9c6089324f81f4671ae7.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 1b6585777b9c8a7031889d672dbf4d81, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675785269084.1b6585777b9c8a7031889d672dbf4d81.', STARTKEY => '60', ENDKEY => ''}] 2023-02-07 15:54:31,100 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-07 15:54:31,100 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675785271100"}]},"ts":"9223372036854775807"} 2023-02-07 15:54:31,102 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-07 15:54:31,113 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:54:31,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 65 msec 2023-02-07 15:54:31,152 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:31,152 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:31,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-07 15:54:31,153 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-07 15:54:31,166 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=411 (was 404) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:46118 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:53010 [Waiting for operation #38] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:56858 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:40292 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:46088 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:42878 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa17a051-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:33550 [Waiting for operation #79] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:42870 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:52614 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:57158 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:40082 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:56844 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:42978 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:39184 [Waiting for operation #37] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:52578 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-76312906_15 at /127.0.0.1:52094 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6ba35714-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=853 (was 847) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=322 (was 324), ProcessCount=173 (was 173), AvailableMemoryMB=3092 (was 3145) 2023-02-07 15:54:31,179 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=411, OpenFileDescriptor=853, MaxFileDescriptor=60000, SystemLoadAverage=322, ProcessCount=173, AvailableMemoryMB=3091 2023-02-07 15:54:31,179 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:54:31,180 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:31,180 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:31,190 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 15:54:31,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 15:54:31,192 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:54:31,194 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 15:54:31,194 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 15:54:31,196 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:31,196 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542 empty. 2023-02-07 15:54:31,197 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:31,197 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-07 15:54:31,206 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-07 15:54:31,207 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => b0c44c23951518cf7a7c968043412542, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp 2023-02-07 15:54:31,214 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:31,214 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing b0c44c23951518cf7a7c968043412542, disabling compactions & flushes 2023-02-07 15:54:31,214 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:31,214 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:31,214 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. after waiting 0 ms 2023-02-07 15:54:31,214 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:31,214 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:31,214 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for b0c44c23951518cf7a7c968043412542: Waiting for close lock at 1675785271214Disabling compacts and flushes for region at 1675785271214Disabling writes for close at 1675785271214Writing region close event to WAL at 1675785271214Closed at 1675785271214 2023-02-07 15:54:31,216 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 15:54:31,217 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785271216"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785271216"}]},"ts":"1675785271216"} 2023-02-07 15:54:31,218 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 15:54:31,219 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 15:54:31,219 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785271219"}]},"ts":"1675785271219"} 2023-02-07 15:54:31,220 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-07 15:54:31,281 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:54:31,282 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:54:31,282 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:54:31,282 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:54:31,282 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 15:54:31,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0c44c23951518cf7a7c968043412542, ASSIGN}] 2023-02-07 15:54:31,283 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0c44c23951518cf7a7c968043412542, ASSIGN 2023-02-07 15:54:31,284 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0c44c23951518cf7a7c968043412542, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,35657,1675785236854; forceNewPlan=false, retain=false 2023-02-07 15:54:31,434 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:54:31,436 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=b0c44c23951518cf7a7c968043412542, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:31,437 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785271436"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785271436"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785271436"}]},"ts":"1675785271436"} 2023-02-07 15:54:31,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure b0c44c23951518cf7a7c968043412542, server=jenkins-hbase12.apache.org,35657,1675785236854}] 2023-02-07 15:54:31,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:31,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b0c44c23951518cf7a7c968043412542, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:54:31,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:31,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:31,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:31,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:31,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:31,596 INFO [StoreOpener-b0c44c23951518cf7a7c968043412542-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:31,597 DEBUG [StoreOpener-b0c44c23951518cf7a7c968043412542-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/cf1 2023-02-07 15:54:31,597 DEBUG [StoreOpener-b0c44c23951518cf7a7c968043412542-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/cf1 2023-02-07 15:54:31,597 INFO [StoreOpener-b0c44c23951518cf7a7c968043412542-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b0c44c23951518cf7a7c968043412542 columnFamilyName cf1 2023-02-07 15:54:31,598 INFO [StoreOpener-b0c44c23951518cf7a7c968043412542-1] regionserver.HStore(310): Store=b0c44c23951518cf7a7c968043412542/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:31,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:31,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:31,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:31,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:54:31,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened b0c44c23951518cf7a7c968043412542; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71660147, jitterRate=0.06781940162181854}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-07 15:54:31,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for b0c44c23951518cf7a7c968043412542: Running coprocessor pre-open hook at 1675785271594Writing region info on filesystem at 1675785271594Initializing all the Stores at 1675785271595 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785271595Cleaning up temporary data from old regions at 1675785271600 (+5 ms)Cleaning up detritus from prior splits at 1675785271600Running coprocessor post-open hooks at 1675785271604 (+4 ms)Region opened successfully at 1675785271604 2023-02-07 15:54:31,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542., pid=87, masterSystemTime=1675785271590 2023-02-07 15:54:31,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:31,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:31,607 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=b0c44c23951518cf7a7c968043412542, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:31,607 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785271607"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785271607"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785271607"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785271607"}]},"ts":"1675785271607"} 2023-02-07 15:54:31,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-07 15:54:31,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure b0c44c23951518cf7a7c968043412542, server=jenkins-hbase12.apache.org,35657,1675785236854 in 170 msec 2023-02-07 15:54:31,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-07 15:54:31,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0c44c23951518cf7a7c968043412542, ASSIGN in 327 msec 2023-02-07 15:54:31,611 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 15:54:31,611 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785271611"}]},"ts":"1675785271611"} 2023-02-07 15:54:31,612 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-07 15:54:31,622 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 15:54:31,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 431 msec 2023-02-07 15:54:31,693 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:31,693 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:31,695 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:54:31,695 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:54:31,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$12(2441): Client=jenkins//136.243.104.168 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-07 15:54:31,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:54:31,723 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-07 15:54:31,723 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-07 15:54:31,723 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-07 15:54:31,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-07 15:54:31,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0c44c23951518cf7a7c968043412542, REOPEN/MOVE}] 2023-02-07 15:54:31,740 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0c44c23951518cf7a7c968043412542, REOPEN/MOVE 2023-02-07 15:54:31,740 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=b0c44c23951518cf7a7c968043412542, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:31,740 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785271740"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785271740"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785271740"}]},"ts":"1675785271740"} 2023-02-07 15:54:31,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure b0c44c23951518cf7a7c968043412542, server=jenkins-hbase12.apache.org,35657,1675785236854}] 2023-02-07 15:54:31,894 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:31,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing b0c44c23951518cf7a7c968043412542, disabling compactions & flushes 2023-02-07 15:54:31,895 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:31,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:31,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. after waiting 0 ms 2023-02-07 15:54:31,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:31,898 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-07 15:54:31,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:54:31,899 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:31,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for b0c44c23951518cf7a7c968043412542: Waiting for close lock at 1675785271894Running coprocessor pre-close hooks at 1675785271894Disabling compacts and flushes for region at 1675785271894Disabling writes for close at 1675785271895 (+1 ms)Writing region close event to WAL at 1675785271895Running coprocessor post-close hooks at 1675785271899 (+4 ms)Closed at 1675785271899 2023-02-07 15:54:31,899 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(3509): Not adding moved region record: b0c44c23951518cf7a7c968043412542 to self. 2023-02-07 15:54:31,900 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:31,901 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=b0c44c23951518cf7a7c968043412542, regionState=CLOSED 2023-02-07 15:54:31,901 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785271901"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785271901"}]},"ts":"1675785271901"} 2023-02-07 15:54:31,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-07 15:54:31,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure b0c44c23951518cf7a7c968043412542, server=jenkins-hbase12.apache.org,35657,1675785236854 in 161 msec 2023-02-07 15:54:31,904 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0c44c23951518cf7a7c968043412542, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase12.apache.org,35657,1675785236854; forceNewPlan=false, retain=true 2023-02-07 15:54:32,054 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:54:32,055 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=b0c44c23951518cf7a7c968043412542, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:32,055 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785272055"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785272055"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785272055"}]},"ts":"1675785272055"} 2023-02-07 15:54:32,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure b0c44c23951518cf7a7c968043412542, server=jenkins-hbase12.apache.org,35657,1675785236854}] 2023-02-07 15:54:32,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:32,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b0c44c23951518cf7a7c968043412542, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:54:32,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:32,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:32,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:32,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:32,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:32,212 INFO [StoreOpener-b0c44c23951518cf7a7c968043412542-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:32,213 DEBUG [StoreOpener-b0c44c23951518cf7a7c968043412542-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/cf1 2023-02-07 15:54:32,213 DEBUG [StoreOpener-b0c44c23951518cf7a7c968043412542-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/cf1 2023-02-07 15:54:32,214 INFO [StoreOpener-b0c44c23951518cf7a7c968043412542-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b0c44c23951518cf7a7c968043412542 columnFamilyName cf1 2023-02-07 15:54:32,214 INFO [StoreOpener-b0c44c23951518cf7a7c968043412542-1] regionserver.HStore(310): Store=b0c44c23951518cf7a7c968043412542/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:54:32,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:32,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:32,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for b0c44c23951518cf7a7c968043412542 2023-02-07 15:54:32,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened b0c44c23951518cf7a7c968043412542; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71164943, jitterRate=0.06044028699398041}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-07 15:54:32,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for b0c44c23951518cf7a7c968043412542: Running coprocessor pre-open hook at 1675785272211Writing region info on filesystem at 1675785272211Initializing all the Stores at 1675785272212 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785272212Cleaning up temporary data from old regions at 1675785272216 (+4 ms)Cleaning up detritus from prior splits at 1675785272216Running coprocessor post-open hooks at 1675785272218 (+2 ms)Region opened successfully at 1675785272218 2023-02-07 15:54:32,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542., pid=92, masterSystemTime=1675785272208 2023-02-07 15:54:32,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:32,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:54:32,221 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=b0c44c23951518cf7a7c968043412542, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:54:32,221 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785272221"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785272221"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785272221"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785272221"}]},"ts":"1675785272221"} 2023-02-07 15:54:32,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-07 15:54:32,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure b0c44c23951518cf7a7c968043412542, server=jenkins-hbase12.apache.org,35657,1675785236854 in 166 msec 2023-02-07 15:54:32,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-07 15:54:32,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0c44c23951518cf7a7c968043412542, REOPEN/MOVE in 484 msec 2023-02-07 15:54:32,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-07 15:54:32,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 500 msec 2023-02-07 15:54:32,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40, ASSIGN}] 2023-02-07 15:54:32,231 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40, ASSIGN 2023-02-07 15:54:32,232 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-07 15:54:32,382 DEBUG [jenkins-hbase12:37073] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:54:32,383 DEBUG [jenkins-hbase12:37073] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:54:32,383 DEBUG [jenkins-hbase12:37073] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:54:32,383 DEBUG [jenkins-hbase12:37073] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:54:32,383 DEBUG [jenkins-hbase12:37073] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-07 15:54:32,384 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=adc6382420bc698237a8a83071387a40, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:54:32,385 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785272384"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1675785272384"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675785272384"}]},"ts":"1675785272384"} 2023-02-07 15:54:32,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure adc6382420bc698237a8a83071387a40, server=jenkins-hbase12.apache.org,37399,1675785236898}] 2023-02-07 15:54:32,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40. 2023-02-07 15:54:32,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => adc6382420bc698237a8a83071387a40, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-07 15:54:32,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:54:32,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica adc6382420bc698237a8a83071387a40 2023-02-07 15:54:32,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:54:32,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for adc6382420bc698237a8a83071387a40 2023-02-07 15:54:32,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for adc6382420bc698237a8a83071387a40 2023-02-07 15:54:32,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:32,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:32,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:32,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:32,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:33,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:33,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:33,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:33,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:33,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:33,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:33,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:33,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:33,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:33,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:34,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:34,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:34,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:34,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:34,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:34,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:34,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:34,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:34,791 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 15:54:34,824 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-07 15:54:34,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:34,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:35,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:35,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:35,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:35,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:35,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:35,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:35,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:35,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:35,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:35,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:36,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:36,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:36,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:36,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:36,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:36,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:36,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:36,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:36,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:36,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:37,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:37,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:37,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:37,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:37,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:37,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:37,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:37,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:37,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:37,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:38,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:38,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:38,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:38,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:38,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:38,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:38,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:38,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:38,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:38,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:39,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:39,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:39,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:39,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:39,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:39,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:39,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:39,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:39,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:39,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:40,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:40,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:40,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:40,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:40,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:40,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:40,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:40,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:40,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:40,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:41,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:41,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:41,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:41,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:41,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:41,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:41,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:41,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:41,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:41,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:42,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:42,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:42,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:42,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:42,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:42,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:42,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:42,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:42,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:42,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:43,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:43,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:43,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:43,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:43,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:43,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:43,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:43,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:43,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:43,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:44,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:44,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:44,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:44,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:44,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:44,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:44,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:44,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:44,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:44,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:45,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:45,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:45,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:45,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:45,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:45,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:45,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:45,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:45,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:45,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:46,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:46,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:46,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:46,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:46,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:46,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:46,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:46,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:46,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:46,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:47,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:47,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:47,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:47,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:47,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:47,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:47,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:47,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:47,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:47,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:48,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:48,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:48,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:48,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:48,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:48,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:48,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:48,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:48,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:48,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:49,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:49,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:49,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:49,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:49,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:49,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:49,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:49,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:49,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:49,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:50,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:50,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:50,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:50,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:50,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:50,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:50,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:50,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:50,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:50,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:51,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:51,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:51,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:51,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:51,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:51,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:51,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:51,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:51,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:51,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:52,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:52,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:52,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:52,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:52,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:52,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:52,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:52,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:52,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:52,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:53,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:53,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:53,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:53,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:53,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:53,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:53,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:53,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:53,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:53,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:54,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:54,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:54,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:54,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:54,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:54,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:54,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:54,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:54,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:54,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:55,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:55,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:55,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:55,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:55,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:55,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:55,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:55,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:55,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:55,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:56,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:56,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:56,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:56,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:56,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:56,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:56,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:56,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:56,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:56,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:57,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:57,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:57,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:57,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:57,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:57,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:57,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:57,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:57,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:57,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:58,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:58,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:58,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:58,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:58,475 DEBUG [regionserver/jenkins-hbase12:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-07 15:54:58,492 DEBUG [regionserver/jenkins-hbase12:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-07 15:54:58,494 DEBUG [regionserver/jenkins-hbase12:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-07 15:54:58,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:58,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:58,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:58,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:58,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:58,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:59,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:59,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:59,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:59,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:59,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:59,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:59,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:59,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:54:59,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:00,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:00,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:00,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:00,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:00,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:00,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:00,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:00,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:00,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:00,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:01,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:01,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:01,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:01,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:01,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:01,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:01,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:01,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:01,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:01,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:02,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:02,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:02,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:02,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:02,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:02,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:02,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:02,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:02,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:02,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:03,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:03,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:03,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:03,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:03,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:03,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:03,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:03,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:03,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:03,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:04,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:04,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:04,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:04,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:04,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:04,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:04,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:04,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:04,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:04,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:05,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:05,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:05,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:05,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:05,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:05,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:05,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:05,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:05,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:05,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:06,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:06,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:06,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:06,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:06,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:06,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:06,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:06,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:06,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:06,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:07,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:07,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:07,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:07,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:07,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:07,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:07,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:07,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:07,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:07,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:08,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:08,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:08,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:08,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:08,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:08,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:08,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:08,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:08,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:08,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:09,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:09,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:09,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:09,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:09,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:09,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:09,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:09,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:09,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:09,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:10,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:10,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:10,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:10,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:10,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:10,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:10,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:10,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:10,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:10,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:11,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:11,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:11,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:11,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:11,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:11,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:11,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:11,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:11,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:11,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:12,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:12,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:12,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:12,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:12,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:12,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:12,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:12,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:12,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:12,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:13,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:13,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:13,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:13,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:13,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:13,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:13,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:13,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:13,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:13,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:14,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:14,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:14,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:14,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:14,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:14,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:14,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:14,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:14,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:14,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:15,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:15,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:15,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:15,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:15,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:15,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:15,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:15,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:15,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:15,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:16,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:16,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:16,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:16,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:16,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:16,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:16,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:16,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:16,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:16,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:17,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:17,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:17,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:17,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:17,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:17,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:17,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:17,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:17,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:17,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:18,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:18,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:18,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:18,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:18,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:18,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:18,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:18,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:18,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:18,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:19,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:19,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:19,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:19,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:19,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:19,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:19,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:19,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:19,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:19,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:20,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:20,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:20,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:20,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:20,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:20,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:20,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:20,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:20,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:20,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:21,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:21,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:21,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:21,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:21,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:21,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:21,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:21,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:21,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:21,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:22,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:22,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:22,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:22,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:22,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:22,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:22,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:22,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:22,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:22,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:23,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:23,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:23,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:23,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:23,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:23,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:23,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:23,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:23,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:23,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:24,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:24,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:24,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:24,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:24,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:24,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:24,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:24,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:24,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:24,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:25,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:25,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:25,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:25,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:25,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:25,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:25,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:25,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:25,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:25,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:26,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:26,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:26,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:26,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:26,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:26,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:26,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:26,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:26,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:26,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:27,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:27,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:27,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:27,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:27,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:27,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:27,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:27,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:27,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:27,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:28,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:28,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:28,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:28,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:28,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:28,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:28,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:28,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:28,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:28,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:29,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:29,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:29,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:29,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:29,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:29,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:29,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:29,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:29,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:29,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:30,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:30,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:30,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:30,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:30,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:30,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:30,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:30,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:30,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:30,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:31,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:31,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:31,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:31,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:31,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:31,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:31,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:31,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:31,820 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:31,820 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:31,824 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:31,825 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:31,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-07 15:55:31,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:31,926 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:31,926 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:31,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-07 15:55:31,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:32,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:32,127 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:32,128 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:32,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-07 15:55:32,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:32,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:32,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:32,430 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:32,430 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:32,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-07 15:55:32,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:32,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:32,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:32,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:32,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:32,933 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:32,934 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:32,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-07 15:55:32,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:33,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:33,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:33,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:33,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:33,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:33,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:33,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:33,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:33,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:33,937 DEBUG [Listener at localhost.localdomain/41965] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:33,938 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:33,939 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:33,940 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:33,940 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:33,943 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting b0c44c23951518cf7a7c968043412542, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:55:33,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 exec-time=9 msec 2023-02-07 15:55:33,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:34,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:34,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 15:55:34,283 INFO [StoreOpener-adc6382420bc698237a8a83071387a40-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region adc6382420bc698237a8a83071387a40 2023-02-07 15:55:34,289 DEBUG [StoreOpener-adc6382420bc698237a8a83071387a40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/cf1 2023-02-07 15:55:34,289 DEBUG [StoreOpener-adc6382420bc698237a8a83071387a40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/cf1 2023-02-07 15:55:34,289 INFO [StoreOpener-adc6382420bc698237a8a83071387a40-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region adc6382420bc698237a8a83071387a40 columnFamilyName cf1 2023-02-07 15:55:34,291 INFO [StoreOpener-adc6382420bc698237a8a83071387a40-1] regionserver.HStore(310): Store=adc6382420bc698237a8a83071387a40/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:34,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened adc6382420bc698237a8a83071387a40; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65521885, jitterRate=-0.02364782989025116}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-07 15:55:34,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for adc6382420bc698237a8a83071387a40: Running coprocessor pre-open hook at 1675785272545Initializing all the Stores at 1675785334282 (+61737 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785334282Running coprocessor post-open hooks at 1675785334294 (+12 ms)Region opened successfully at 1675785334294 2023-02-07 15:55:34,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40., pid=94, masterSystemTime=1675785272540 2023-02-07 15:55:34,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40. 2023-02-07 15:55:34,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40. 2023-02-07 15:55:34,297 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=adc6382420bc698237a8a83071387a40, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:34,297 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785334297"},{"qualifier":"server_0001","vlen":32,"tag":[],"timestamp":"1675785334297"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1675785334297"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1675785334297"}]},"ts":"1675785334297"} 2023-02-07 15:55:34,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-07 15:55:34,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure adc6382420bc698237a8a83071387a40, server=jenkins-hbase12.apache.org,37399,1675785236898 in 1 mins, 1.913 sec 2023-02-07 15:55:34,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-07 15:55:34,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40, ASSIGN in 1 mins, 2.072 sec 2023-02-07 15:55:34,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.6 sec 2023-02-07 15:55:34,307 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1675785271699","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62607,"client":"136.243.104.168:58182","queuetimems":0,"class":"HMaster"} 2023-02-07 15:55:34,307 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 607, skipping over this response of 3 bytes 2023-02-07 15:55:36,190 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up 2023-02-07 15:55:36,192 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:55:36,192 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:36,192 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:36,194 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:36,195 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:36,195 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:36,196 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:36,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:36,198 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:36,198 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:36,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-07 15:55:36,199 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785336199"}]},"ts":"1675785336199"} 2023-02-07 15:55:36,200 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-07 15:55:36,277 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-07 15:55:36,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0c44c23951518cf7a7c968043412542, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40, UNASSIGN}] 2023-02-07 15:55:36,279 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40, UNASSIGN 2023-02-07 15:55:36,280 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=adc6382420bc698237a8a83071387a40, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:36,281 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785336280"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1675785336280"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675785336280"}]},"ts":"1675785336280"} 2023-02-07 15:55:36,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure adc6382420bc698237a8a83071387a40, server=jenkins-hbase12.apache.org,37399,1675785236898}] 2023-02-07 15:55:36,287 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0c44c23951518cf7a7c968043412542, UNASSIGN 2023-02-07 15:55:36,289 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=b0c44c23951518cf7a7c968043412542, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:36,289 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785336289"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785336289"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785336289"}]},"ts":"1675785336289"} 2023-02-07 15:55:36,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure b0c44c23951518cf7a7c968043412542, server=jenkins-hbase12.apache.org,35657,1675785236854}] 2023-02-07 15:55:36,299 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:36,299 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:36,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-07 15:55:36,436 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close adc6382420bc698237a8a83071387a40 2023-02-07 15:55:36,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing adc6382420bc698237a8a83071387a40, disabling compactions & flushes 2023-02-07 15:55:36,437 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40. 2023-02-07 15:55:36,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40. 2023-02-07 15:55:36,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40. after waiting 0 ms 2023-02-07 15:55:36,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40. 2023-02-07 15:55:36,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:55:36,439 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40. 2023-02-07 15:55:36,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for adc6382420bc698237a8a83071387a40: Waiting for close lock at 1675785336436Running coprocessor pre-close hooks at 1675785336436Disabling compacts and flushes for region at 1675785336436Disabling writes for close at 1675785336438 (+2 ms)Writing region close event to WAL at 1675785336438Running coprocessor post-close hooks at 1675785336438Closed at 1675785336439 (+1 ms) 2023-02-07 15:55:36,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed adc6382420bc698237a8a83071387a40 2023-02-07 15:55:36,442 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=adc6382420bc698237a8a83071387a40, regionState=CLOSED 2023-02-07 15:55:36,442 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785336442"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1675785336442"}]},"ts":"1675785336442"} 2023-02-07 15:55:36,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close b0c44c23951518cf7a7c968043412542 2023-02-07 15:55:36,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing b0c44c23951518cf7a7c968043412542, disabling compactions & flushes 2023-02-07 15:55:36,444 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:55:36,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:55:36,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. after waiting 0 ms 2023-02-07 15:55:36,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:55:36,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-07 15:55:36,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure adc6382420bc698237a8a83071387a40, server=jenkins-hbase12.apache.org,37399,1675785236898 in 162 msec 2023-02-07 15:55:36,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-07 15:55:36,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40, UNASSIGN in 171 msec 2023-02-07 15:55:36,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:55:36,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542. 2023-02-07 15:55:36,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for b0c44c23951518cf7a7c968043412542: Waiting for close lock at 1675785336443Running coprocessor pre-close hooks at 1675785336443Disabling compacts and flushes for region at 1675785336443Disabling writes for close at 1675785336444 (+1 ms)Writing region close event to WAL at 1675785336445 (+1 ms)Running coprocessor post-close hooks at 1675785336450 (+5 ms)Closed at 1675785336451 (+1 ms) 2023-02-07 15:55:36,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed b0c44c23951518cf7a7c968043412542 2023-02-07 15:55:36,454 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=b0c44c23951518cf7a7c968043412542, regionState=CLOSED 2023-02-07 15:55:36,454 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675785336454"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785336454"}]},"ts":"1675785336454"} 2023-02-07 15:55:36,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-07 15:55:36,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure b0c44c23951518cf7a7c968043412542, server=jenkins-hbase12.apache.org,35657,1675785236854 in 165 msec 2023-02-07 15:55:36,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-07 15:55:36,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b0c44c23951518cf7a7c968043412542, UNASSIGN in 180 msec 2023-02-07 15:55:36,460 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785336460"}]},"ts":"1675785336460"} 2023-02-07 15:55:36,461 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-07 15:55:36,469 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-07 15:55:36,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 274 msec 2023-02-07 15:55:36,500 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:36,500 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:36,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-07 15:55:36,501 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-07 15:55:36,501 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:36,501 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:36,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:36,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:36,504 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:36,504 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:36,504 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:36,504 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:36,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-07 15:55:36,510 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542 2023-02-07 15:55:36,512 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/recovered.edits] 2023-02-07 15:55:36,516 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/recovered.edits/7.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542/recovered.edits/7.seqid 2023-02-07 15:55:36,517 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/b0c44c23951518cf7a7c968043412542 2023-02-07 15:55:36,517 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-07 15:55:36,519 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:36,520 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-07 15:55:36,522 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-07 15:55:36,523 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:36,523 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-07 15:55:36,523 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785336523"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:36,523 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785336523"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:36,525 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-07 15:55:36,526 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b0c44c23951518cf7a7c968043412542, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675785271191.b0c44c23951518cf7a7c968043412542.', STARTKEY => '', ENDKEY => ''}, {ENCODED => adc6382420bc698237a8a83071387a40, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675785271191_0001.adc6382420bc698237a8a83071387a40.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-07 15:55:36,526 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-07 15:55:36,526 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675785336526"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:36,528 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-07 15:55:36,541 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:36,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 39 msec 2023-02-07 15:55:36,605 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:36,605 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:36,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-07 15:55:36,606 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-07 15:55:36,618 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=348 (was 411), OpenFileDescriptor=655 (was 853), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=213 (was 322), ProcessCount=174 (was 173) - ProcessCount LEAK? -, AvailableMemoryMB=3108 (was 3091) - AvailableMemoryMB LEAK? - 2023-02-07 15:55:36,630 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=349, OpenFileDescriptor=655, MaxFileDescriptor=60000, SystemLoadAverage=213, ProcessCount=174, AvailableMemoryMB=3106 2023-02-07 15:55:36,631 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:55:36,631 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:36,631 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:36,637 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 15:55:36,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 15:55:36,639 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:36,640 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 15:55:36,641 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 15:55:36,642 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:36,643 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da empty. 2023-02-07 15:55:36,643 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:36,643 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-07 15:55:36,658 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-07 15:55:36,659 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 784b8ca3a006eb051cd4eb81274042da, NAME => 'testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp 2023-02-07 15:55:36,672 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:36,672 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 784b8ca3a006eb051cd4eb81274042da, disabling compactions & flushes 2023-02-07 15:55:36,672 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:36,672 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:36,672 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. after waiting 0 ms 2023-02-07 15:55:36,672 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:36,672 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:36,673 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 784b8ca3a006eb051cd4eb81274042da: Waiting for close lock at 1675785336672Disabling compacts and flushes for region at 1675785336672Disabling writes for close at 1675785336672Writing region close event to WAL at 1675785336672Closed at 1675785336672 2023-02-07 15:55:36,681 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 15:55:36,682 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675785336682"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785336682"}]},"ts":"1675785336682"} 2023-02-07 15:55:36,683 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 15:55:36,683 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 15:55:36,684 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785336683"}]},"ts":"1675785336683"} 2023-02-07 15:55:36,684 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-07 15:55:36,700 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:55:36,702 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:55:36,702 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:55:36,702 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:55:36,702 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 15:55:36,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=784b8ca3a006eb051cd4eb81274042da, ASSIGN}] 2023-02-07 15:55:36,704 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=784b8ca3a006eb051cd4eb81274042da, ASSIGN 2023-02-07 15:55:36,704 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=784b8ca3a006eb051cd4eb81274042da, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:55:36,855 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:55:36,856 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=784b8ca3a006eb051cd4eb81274042da, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:36,856 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675785336856"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785336856"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785336856"}]},"ts":"1675785336856"} 2023-02-07 15:55:36,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 784b8ca3a006eb051cd4eb81274042da, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:37,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:37,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 784b8ca3a006eb051cd4eb81274042da, NAME => 'testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:55:37,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:55:37,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:37,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:37,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:37,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:37,019 INFO [StoreOpener-784b8ca3a006eb051cd4eb81274042da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:37,020 DEBUG [StoreOpener-784b8ca3a006eb051cd4eb81274042da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1 2023-02-07 15:55:37,020 DEBUG [StoreOpener-784b8ca3a006eb051cd4eb81274042da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1 2023-02-07 15:55:37,020 INFO [StoreOpener-784b8ca3a006eb051cd4eb81274042da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 784b8ca3a006eb051cd4eb81274042da columnFamilyName cf1 2023-02-07 15:55:37,021 INFO [StoreOpener-784b8ca3a006eb051cd4eb81274042da-1] regionserver.HStore(310): Store=784b8ca3a006eb051cd4eb81274042da/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:37,021 INFO [StoreOpener-784b8ca3a006eb051cd4eb81274042da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:37,022 DEBUG [StoreOpener-784b8ca3a006eb051cd4eb81274042da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2 2023-02-07 15:55:37,022 DEBUG [StoreOpener-784b8ca3a006eb051cd4eb81274042da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2 2023-02-07 15:55:37,022 INFO [StoreOpener-784b8ca3a006eb051cd4eb81274042da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 784b8ca3a006eb051cd4eb81274042da columnFamilyName cf2 2023-02-07 15:55:37,022 INFO [StoreOpener-784b8ca3a006eb051cd4eb81274042da-1] regionserver.HStore(310): Store=784b8ca3a006eb051cd4eb81274042da/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:37,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:37,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:37,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:55:37,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:37,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:55:37,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 784b8ca3a006eb051cd4eb81274042da; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68983025, jitterRate=0.027927175164222717}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:55:37,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 784b8ca3a006eb051cd4eb81274042da: Running coprocessor pre-open hook at 1675785337018Writing region info on filesystem at 1675785337018Initializing all the Stores at 1675785337019 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785337019Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785337019Cleaning up temporary data from old regions at 1675785337024 (+5 ms)Cleaning up detritus from prior splits at 1675785337024Running coprocessor post-open hooks at 1675785337028 (+4 ms)Region opened successfully at 1675785337028 2023-02-07 15:55:37,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da., pid=104, masterSystemTime=1675785337013 2023-02-07 15:55:37,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:37,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:37,031 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=784b8ca3a006eb051cd4eb81274042da, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,031 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675785337031"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785337031"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785337031"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785337031"}]},"ts":"1675785337031"} 2023-02-07 15:55:37,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-07 15:55:37,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 784b8ca3a006eb051cd4eb81274042da, server=jenkins-hbase12.apache.org,38629,1675785236797 in 174 msec 2023-02-07 15:55:37,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-07 15:55:37,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=784b8ca3a006eb051cd4eb81274042da, ASSIGN in 332 msec 2023-02-07 15:55:37,038 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 15:55:37,038 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785337038"}]},"ts":"1675785337038"} 2023-02-07 15:55:37,039 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-07 15:55:37,091 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 15:55:37,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 453 msec 2023-02-07 15:55:37,147 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:37,147 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:37,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:37,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-07 15:55:37,150 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-07 15:55:37,150 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,150 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-07 15:55:37,150 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-07 15:55:37,150 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,150 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,161 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,161 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,161 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,161 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,162 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,162 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:37,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,163 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:37,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:37,163 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:37,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,163 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,163 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:37,163 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:37,163 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:37,163 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:37,164 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. started... 2023-02-07 15:55:37,164 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 784b8ca3a006eb051cd4eb81274042da 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 15:55:37,174 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,174 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/9543a6949cd64c059bc0ae28e646390c 2023-02-07 15:55:37,184 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,184 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,184 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:37,185 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,185 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:37,186 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/e88415910409439ead995a49120ccfba 2023-02-07 15:55:37,190 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/9543a6949cd64c059bc0ae28e646390c as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/9543a6949cd64c059bc0ae28e646390c 2023-02-07 15:55:37,194 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/9543a6949cd64c059bc0ae28e646390c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:55:37,195 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/e88415910409439ead995a49120ccfba as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/e88415910409439ead995a49120ccfba 2023-02-07 15:55:37,198 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/e88415910409439ead995a49120ccfba, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:55:37,199 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 784b8ca3a006eb051cd4eb81274042da in 35ms, sequenceid=6, compaction requested=false 2023-02-07 15:55:37,199 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-07 15:55:37,200 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 784b8ca3a006eb051cd4eb81274042da: 2023-02-07 15:55:37,200 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:37,200 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:37,200 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:37,200 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,200 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:37,200 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:37,205 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,205 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:37,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:37,206 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,206 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:37,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:37,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:37,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:37,207 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-07 15:55:37,207 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73d29f43[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:37,207 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-07 15:55:37,207 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,216 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,216 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,216 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,216 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,216 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:37,216 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:37,216 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:37,216 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:37,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,217 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:37,217 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:37,217 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:37,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,217 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:37,217 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:37,217 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:37,227 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:37,227 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:37,227 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:37,227 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:37,227 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:37,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,227 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:37,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:37,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:37,237 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:37,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:37,237 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:37,237 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:37,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:37,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:37,245 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-07 15:55:37,245 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-07 15:55:37,245 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:37,245 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:37,245 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-07 15:55:37,245 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:37,258 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,258 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,258 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,258 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,258 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,258 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,259 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:37,259 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,260 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,261 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:37,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,263 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:37,264 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,264 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,277 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,277 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,278 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,277 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,278 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,278 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:37,277 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,278 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,278 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,278 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,278 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,278 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-07 15:55:37,278 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:37,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,278 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:37,279 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:37,277 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,278 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,279 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,279 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,279 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,279 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,279 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,379 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:37,379 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:37,379 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:37,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:37,404 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:37,404 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:37,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:37,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:37,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-07 15:55:37,407 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-07 15:55:37,407 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,407 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-07 15:55:37,407 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-07 15:55:37,408 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,408 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,445 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,445 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,445 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,445 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,446 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,446 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:37,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,448 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,448 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:37,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,449 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:37,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:37,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,449 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:37,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:37,450 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:37,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,450 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:37,450 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:37,450 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:37,450 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:37,450 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:37,451 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,450 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:37,451 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:37,450 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,451 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:37,451 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:37,451 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:37,451 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:37,451 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:37,452 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:37,451 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:37,452 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,452 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:37,451 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:37,452 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. started... 2023-02-07 15:55:37,452 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:37,452 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:37,452 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:37,452 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 784b8ca3a006eb051cd4eb81274042da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:37,465 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/303b6a51dbc74a12816a4b17dca558a7 2023-02-07 15:55:37,469 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,477 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/4a8b7e3b08eb4a14804a153bed57e524 2023-02-07 15:55:37,479 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,479 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,479 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:37,480 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,480 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:37,481 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/303b6a51dbc74a12816a4b17dca558a7 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/303b6a51dbc74a12816a4b17dca558a7 2023-02-07 15:55:37,485 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/303b6a51dbc74a12816a4b17dca558a7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:55:37,486 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/4a8b7e3b08eb4a14804a153bed57e524 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/4a8b7e3b08eb4a14804a153bed57e524 2023-02-07 15:55:37,490 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/4a8b7e3b08eb4a14804a153bed57e524, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:55:37,491 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 784b8ca3a006eb051cd4eb81274042da in 39ms, sequenceid=19, compaction requested=false 2023-02-07 15:55:37,491 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 784b8ca3a006eb051cd4eb81274042da: 2023-02-07 15:55:37,491 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:37,491 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:37,491 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:37,491 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,491 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:37,491 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:37,500 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,500 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:37,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:37,501 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:37,501 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:37,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:37,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:37,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-07 15:55:37,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b8439ed[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:37,503 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-07 15:55:37,503 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,511 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,511 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,511 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,511 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:37,511 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:37,511 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:37,511 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,511 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:37,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,511 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:37,511 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:37,512 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:37,512 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:37,512 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:37,512 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:37,521 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:37,521 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,521 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:37,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:37,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:37,522 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:37,532 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:37,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:37,532 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:37,532 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:37,532 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:37,532 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:37,532 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:37,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:37,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:37,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:37,534 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-07 15:55:37,534 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-07 15:55:37,534 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:37,534 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:37,534 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-07 15:55:37,534 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:37,542 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,542 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,542 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,542 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,542 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,542 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,543 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:37,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:37,543 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,543 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:37,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,544 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,545 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,545 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:37,545 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,553 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,553 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,553 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,553 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,553 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,553 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,553 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,553 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,553 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,553 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-07 15:55:37,554 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:37,553 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:37,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,554 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,554 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,554 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,554 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,554 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,554 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,554 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,554 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:37,554 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:37,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,654 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:37,655 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:37,655 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:37,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:37,668 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:37,669 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:37,669 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:37,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:37,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-07 15:55:37,671 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-07 15:55:37,671 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,671 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-07 15:55:37,671 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-07 15:55:37,672 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,672 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,700 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,700 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,700 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,700 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,701 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,701 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:37,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,702 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,702 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:37,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:37,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:37,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:37,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:37,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:37,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:37,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,703 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,703 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:37,703 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:37,703 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,703 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:37,703 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:37,703 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:37,703 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. started... 2023-02-07 15:55:37,704 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 784b8ca3a006eb051cd4eb81274042da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:37,711 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,711 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,711 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,711 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:37,711 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,711 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:37,719 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/29de069bf9bd41bb846364d9b1b11661 2023-02-07 15:55:37,730 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/65385e7abe334de7a1eb4147e3767fcc 2023-02-07 15:55:37,734 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/29de069bf9bd41bb846364d9b1b11661 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/29de069bf9bd41bb846364d9b1b11661 2023-02-07 15:55:37,737 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/29de069bf9bd41bb846364d9b1b11661, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:55:37,737 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/65385e7abe334de7a1eb4147e3767fcc as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/65385e7abe334de7a1eb4147e3767fcc 2023-02-07 15:55:37,740 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/65385e7abe334de7a1eb4147e3767fcc, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:55:37,741 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 784b8ca3a006eb051cd4eb81274042da in 37ms, sequenceid=32, compaction requested=true 2023-02-07 15:55:37,741 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 784b8ca3a006eb051cd4eb81274042da: 2023-02-07 15:55:37,741 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:37,741 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:37,741 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:37,741 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,741 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:37,741 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:37,753 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,753 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:37,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:37,753 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,754 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:37,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:37,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:37,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:37,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-07 15:55:37,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fcfa117[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:37,755 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-07 15:55:37,755 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,763 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,763 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,763 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,764 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:37,764 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,764 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:37,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,764 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:37,764 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:37,764 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:37,764 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:37,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,764 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:37,764 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:37,764 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:37,764 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:37,774 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:37,774 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:37,774 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:37,774 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:37,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:37,784 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:37,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:37,784 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:37,784 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:37,784 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:37,784 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:37,785 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:37,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:37,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:37,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-07 15:55:37,788 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-07 15:55:37,788 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:37,788 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:37,788 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-07 15:55:37,788 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:37,795 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,795 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,795 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,795 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,795 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,795 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,795 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:37,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:37,795 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:37,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,796 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,796 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:37,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:37,797 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:37,798 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,798 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,835 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,835 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,836 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,836 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:37,835 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,835 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,836 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:37,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,835 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,836 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-07 15:55:37,836 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:37,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,836 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,836 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:37,836 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,836 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:37,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:37,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,836 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:37,837 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,837 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:37,837 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,837 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:37,837 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,837 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,837 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,937 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:37,937 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:37,937 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:37,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:37,948 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:37,948 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:37,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:37,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:37,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-07 15:55:37,951 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-07 15:55:37,951 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,951 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-07 15:55:37,951 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-07 15:55:37,951 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,951 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,972 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,972 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,972 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,972 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:37,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:37,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:37,972 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:37,973 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:37,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,973 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,973 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,973 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:37,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:37,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:37,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,974 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:37,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:37,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:37,974 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:37,974 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:37,974 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:37,974 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:37,974 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:37,974 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:37,974 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:37,974 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:37,974 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:37,974 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,975 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:37,975 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,974 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:37,975 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:37,975 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:37,975 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:37,975 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:37,975 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:37,975 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:37,975 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:37,975 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. started... 2023-02-07 15:55:37,975 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:37,975 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:37,975 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:37,975 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 784b8ca3a006eb051cd4eb81274042da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:37,975 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:37,985 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,985 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/ece56697753c4f3b8c23dda9634efd2f 2023-02-07 15:55:37,995 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,995 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:37,995 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,996 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:37,996 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:38,001 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/be037cd5c2974a5890fd22d4cd475f9d 2023-02-07 15:55:38,007 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/ece56697753c4f3b8c23dda9634efd2f as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/ece56697753c4f3b8c23dda9634efd2f 2023-02-07 15:55:38,012 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/ece56697753c4f3b8c23dda9634efd2f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:55:38,013 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/be037cd5c2974a5890fd22d4cd475f9d as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/be037cd5c2974a5890fd22d4cd475f9d 2023-02-07 15:55:38,019 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/be037cd5c2974a5890fd22d4cd475f9d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:55:38,020 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 784b8ca3a006eb051cd4eb81274042da in 45ms, sequenceid=45, compaction requested=true 2023-02-07 15:55:38,020 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 784b8ca3a006eb051cd4eb81274042da: 2023-02-07 15:55:38,020 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:38,020 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:38,020 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:38,020 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,020 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:38,020 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:38,035 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,035 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:38,036 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,036 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:38,038 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-07 15:55:38,038 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3332f8e3[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:38,038 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-07 15:55:38,038 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,048 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,048 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,048 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,048 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,048 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:38,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,048 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:38,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,048 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:38,048 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:38,048 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:38,048 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:38,048 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:38,048 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:38,048 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:38,049 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:38,058 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:38,058 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:38,058 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:38,069 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:38,069 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,069 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:38,069 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:38,069 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:38,069 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:38,069 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:38,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:38,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:38,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:38,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:38,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:38,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:38,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-07 15:55:38,073 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-07 15:55:38,073 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:38,073 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:38,073 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-07 15:55:38,073 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:38,079 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,079 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,079 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,079 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,079 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,079 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,080 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,079 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,080 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,080 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,081 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,081 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,082 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:38,100 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,100 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,100 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,100 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-07 15:55:38,101 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:38,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,100 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,100 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:38,100 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,101 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:38,101 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:38,100 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:38,100 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,101 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,101 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,101 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:38,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,101 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,102 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,102 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,102 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,102 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,102 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testInvalidSplitKey because node does not exist (not an error) 2023-02-07 15:55:38,201 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:38,201 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:38,201 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:38,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:38,213 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:38,214 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:38,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:38,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:38,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-07 15:55:38,216 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-07 15:55:38,216 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:38,216 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-07 15:55:38,216 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-07 15:55:38,216 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,216 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,227 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,227 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,227 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:38,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:38,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:38,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:38,228 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:38,229 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:38,229 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:38,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:38,229 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:38,229 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:38,229 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. started... 2023-02-07 15:55:38,230 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 784b8ca3a006eb051cd4eb81274042da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:38,237 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,239 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/b2279cb2e5d241c0a5befda57be7fabc 2023-02-07 15:55:38,248 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,248 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,248 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:38,248 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,248 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:38,250 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/fec09a6be43b41019cb9003033ea11a5 2023-02-07 15:55:38,254 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/b2279cb2e5d241c0a5befda57be7fabc as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/b2279cb2e5d241c0a5befda57be7fabc 2023-02-07 15:55:38,259 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/b2279cb2e5d241c0a5befda57be7fabc, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:55:38,260 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/fec09a6be43b41019cb9003033ea11a5 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/fec09a6be43b41019cb9003033ea11a5 2023-02-07 15:55:38,265 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/fec09a6be43b41019cb9003033ea11a5, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:55:38,266 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 784b8ca3a006eb051cd4eb81274042da in 36ms, sequenceid=58, compaction requested=true 2023-02-07 15:55:38,266 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 784b8ca3a006eb051cd4eb81274042da: 2023-02-07 15:55:38,266 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:38,266 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:38,266 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:38,266 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,266 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:38,266 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:38,311 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,311 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:38,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:38,311 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,311 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:38,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:38,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:38,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:38,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-07 15:55:38,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54fd4141[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:38,313 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-07 15:55:38,313 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,363 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,363 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,363 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,364 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:38,364 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,364 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:38,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,364 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:38,364 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:38,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,364 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:38,364 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:38,364 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:38,364 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:38,364 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:38,364 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:38,384 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:38,385 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:38,385 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:38,395 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,395 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:38,395 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:38,395 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:38,395 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:38,396 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:38,396 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:38,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:38,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:38,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:38,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:38,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:38,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:38,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-07 15:55:38,401 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-07 15:55:38,401 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:38,401 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:38,401 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-07 15:55:38,401 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:38,414 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,414 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:38,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:38,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,414 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,414 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,414 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,415 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:38,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,415 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,416 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,416 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:38,417 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:38,420 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,420 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,421 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,435 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,435 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,435 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,435 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,435 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,436 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:38,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-07 15:55:38,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:38,436 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,436 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,436 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,435 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,437 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,437 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:38,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,437 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:38,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,437 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,438 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,438 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,438 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,438 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,438 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,438 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,538 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:38,538 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:38,538 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:38,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:38,550 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:38,550 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:38,551 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:38,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:38,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-07 15:55:38,552 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-07 15:55:38,552 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:38,552 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-07 15:55:38,552 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-07 15:55:38,553 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,553 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,563 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,563 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,563 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,563 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,564 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,564 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:38,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,565 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:38,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 15:55:38,566 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:38,566 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:38,566 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 15:55:38,566 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:38,566 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:38,566 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:38,566 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 15:55:38,566 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:38,566 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:38,566 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:38,566 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 15:55:38,566 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:38,566 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:38,566 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:38,566 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:38,566 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:38,566 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 15:55:38,567 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:38,567 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:38,567 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:38,567 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:38,567 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. started... 2023-02-07 15:55:38,567 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:38,567 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 784b8ca3a006eb051cd4eb81274042da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:38,574 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,577 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/9be42171ce0947ef83383dbef3952908 2023-02-07 15:55:38,584 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,584 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,585 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:38,585 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,585 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:38,589 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/f30fcac4934c4f35bfef9905e5c88809 2023-02-07 15:55:38,594 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/9be42171ce0947ef83383dbef3952908 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/9be42171ce0947ef83383dbef3952908 2023-02-07 15:55:38,599 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/9be42171ce0947ef83383dbef3952908, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:55:38,600 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/f30fcac4934c4f35bfef9905e5c88809 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/f30fcac4934c4f35bfef9905e5c88809 2023-02-07 15:55:38,605 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/f30fcac4934c4f35bfef9905e5c88809, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:55:38,606 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 784b8ca3a006eb051cd4eb81274042da in 39ms, sequenceid=71, compaction requested=true 2023-02-07 15:55:38,606 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 784b8ca3a006eb051cd4eb81274042da: 2023-02-07 15:55:38,606 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:38,607 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:38,607 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:38,607 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,607 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 15:55:38,607 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 15:55:38,616 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,616 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:38,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:38,617 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,617 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:38,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:38,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:38,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:38,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-07 15:55:38,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ecc5162[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:38,620 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-07 15:55:38,620 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,627 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,627 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,627 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,627 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:38,627 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,627 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:38,627 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:38,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,627 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:38,627 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:38,627 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:38,627 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:38,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,627 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 15:55:38,627 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 15:55:38,627 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:38,637 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:38,637 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,637 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:38,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:38,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:38,637 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:38,637 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:38,637 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:38,637 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:38,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:38,648 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 15:55:38,648 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:38,648 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 15:55:38,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:38,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:38,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:38,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-07 15:55:38,652 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-07 15:55:38,652 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:38,652 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:38,652 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-07 15:55:38,652 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:38,658 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,658 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,658 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,658 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,658 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,659 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:38,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:38,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,659 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,659 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:38,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,660 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,661 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:38,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,663 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:38,664 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 15:55:38,665 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,679 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,679 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,679 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,679 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,680 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,679 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:38,680 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,680 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-07 15:55:38,680 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:38,680 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:38,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,680 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,681 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:38,681 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:38,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,680 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,681 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 15:55:38,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,680 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:38,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:38,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:38,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,681 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:38,681 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,682 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:38,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:38,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:38,682 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,682 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 15:55:38,682 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 15:55:38,781 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:38,781 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:38,781 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:38,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:38,793 INFO [Listener at localhost.localdomain/41965] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,795 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/303b6a51dbc74a12816a4b17dca558a7 because midkey is the same as first or last row 2023-02-07 15:55:38,795 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/4a8b7e3b08eb4a14804a153bed57e524 because midkey is the same as first or last row 2023-02-07 15:55:38,795 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.HRegion(2744): Flushing 784b8ca3a006eb051cd4eb81274042da 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 15:55:38,804 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/8c622c76741843929af020319e29ce9e 2023-02-07 15:55:38,818 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/4adfe713febc4e26b43fc141c75bd20e 2023-02-07 15:55:38,822 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf1/8c622c76741843929af020319e29ce9e as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/8c622c76741843929af020319e29ce9e 2023-02-07 15:55:38,826 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/8c622c76741843929af020319e29ce9e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:55:38,826 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/.tmp/cf2/4adfe713febc4e26b43fc141c75bd20e as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/4adfe713febc4e26b43fc141c75bd20e 2023-02-07 15:55:38,829 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/4adfe713febc4e26b43fc141c75bd20e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:55:38,830 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 784b8ca3a006eb051cd4eb81274042da in 35ms, sequenceid=82, compaction requested=true 2023-02-07 15:55:38,831 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.HRegion(2445): Flush status journal for 784b8ca3a006eb051cd4eb81274042da: 2023-02-07 15:55:38,831 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/303b6a51dbc74a12816a4b17dca558a7 because midkey is the same as first or last row 2023-02-07 15:55:38,831 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38629] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/4a8b7e3b08eb4a14804a153bed57e524 because midkey is the same as first or last row 2023-02-07 15:55:38,832 DEBUG [Listener at localhost.localdomain/41965] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,832 DEBUG [Listener at localhost.localdomain/41965] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-07 15:55:38,832 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:55:38,832 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:38,832 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:38,833 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:38,833 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:38,834 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-07 15:55:38,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testInvalidSplitKey 2023-02-07 15:55:38,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:38,837 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:38,837 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:38,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-07 15:55:38,838 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785338838"}]},"ts":"1675785338838"} 2023-02-07 15:55:38,839 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-07 15:55:38,848 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-07 15:55:38,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=784b8ca3a006eb051cd4eb81274042da, UNASSIGN}] 2023-02-07 15:55:38,849 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=784b8ca3a006eb051cd4eb81274042da, UNASSIGN 2023-02-07 15:55:38,850 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=784b8ca3a006eb051cd4eb81274042da, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:38,850 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675785338850"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785338850"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785338850"}]},"ts":"1675785338850"} 2023-02-07 15:55:38,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 784b8ca3a006eb051cd4eb81274042da, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:38,938 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:38,938 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:38,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-07 15:55:39,002 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:39,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 784b8ca3a006eb051cd4eb81274042da, disabling compactions & flushes 2023-02-07 15:55:39,002 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:39,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:39,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. after waiting 0 ms 2023-02-07 15:55:39,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:39,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 15:55:39,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:55:39,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da. 2023-02-07 15:55:39,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 784b8ca3a006eb051cd4eb81274042da: Waiting for close lock at 1675785339002Running coprocessor pre-close hooks at 1675785339002Disabling compacts and flushes for region at 1675785339002Disabling writes for close at 1675785339002Writing region close event to WAL at 1675785339003 (+1 ms)Running coprocessor post-close hooks at 1675785339006 (+3 ms)Closed at 1675785339006 2023-02-07 15:55:39,007 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:39,007 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=784b8ca3a006eb051cd4eb81274042da, regionState=CLOSED 2023-02-07 15:55:39,007 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675785339007"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785339007"}]},"ts":"1675785339007"} 2023-02-07 15:55:39,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-07 15:55:39,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 784b8ca3a006eb051cd4eb81274042da, server=jenkins-hbase12.apache.org,38629,1675785236797 in 157 msec 2023-02-07 15:55:39,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-07 15:55:39,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=784b8ca3a006eb051cd4eb81274042da, UNASSIGN in 161 msec 2023-02-07 15:55:39,011 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785339011"}]},"ts":"1675785339011"} 2023-02-07 15:55:39,011 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-07 15:55:39,016 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-07 15:55:39,018 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 183 msec 2023-02-07 15:55:39,139 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:39,139 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:39,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-07 15:55:39,140 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-07 15:55:39,140 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:39,141 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:39,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testInvalidSplitKey 2023-02-07 15:55:39,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:39,143 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:39,143 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:39,144 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:39,144 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:39,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-07 15:55:39,146 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:39,148 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/recovered.edits] 2023-02-07 15:55:39,151 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/29de069bf9bd41bb846364d9b1b11661 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/29de069bf9bd41bb846364d9b1b11661 2023-02-07 15:55:39,153 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/303b6a51dbc74a12816a4b17dca558a7 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/303b6a51dbc74a12816a4b17dca558a7 2023-02-07 15:55:39,154 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/8c622c76741843929af020319e29ce9e to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/8c622c76741843929af020319e29ce9e 2023-02-07 15:55:39,156 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/9543a6949cd64c059bc0ae28e646390c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/9543a6949cd64c059bc0ae28e646390c 2023-02-07 15:55:39,157 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/9be42171ce0947ef83383dbef3952908 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/9be42171ce0947ef83383dbef3952908 2023-02-07 15:55:39,159 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/b2279cb2e5d241c0a5befda57be7fabc to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/b2279cb2e5d241c0a5befda57be7fabc 2023-02-07 15:55:39,160 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/ece56697753c4f3b8c23dda9634efd2f to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf1/ece56697753c4f3b8c23dda9634efd2f 2023-02-07 15:55:39,163 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/4a8b7e3b08eb4a14804a153bed57e524 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/4a8b7e3b08eb4a14804a153bed57e524 2023-02-07 15:55:39,164 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/4adfe713febc4e26b43fc141c75bd20e to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/4adfe713febc4e26b43fc141c75bd20e 2023-02-07 15:55:39,166 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/65385e7abe334de7a1eb4147e3767fcc to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/65385e7abe334de7a1eb4147e3767fcc 2023-02-07 15:55:39,168 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/be037cd5c2974a5890fd22d4cd475f9d to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/be037cd5c2974a5890fd22d4cd475f9d 2023-02-07 15:55:39,169 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/e88415910409439ead995a49120ccfba to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/e88415910409439ead995a49120ccfba 2023-02-07 15:55:39,172 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/f30fcac4934c4f35bfef9905e5c88809 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/f30fcac4934c4f35bfef9905e5c88809 2023-02-07 15:55:39,173 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/fec09a6be43b41019cb9003033ea11a5 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/cf2/fec09a6be43b41019cb9003033ea11a5 2023-02-07 15:55:39,177 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/recovered.edits/85.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da/recovered.edits/85.seqid 2023-02-07 15:55:39,177 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testInvalidSplitKey/784b8ca3a006eb051cd4eb81274042da 2023-02-07 15:55:39,177 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-07 15:55:39,179 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:39,180 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-07 15:55:39,184 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-07 15:55:39,184 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:39,185 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-07 15:55:39,185 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785339185"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:39,186 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-07 15:55:39,186 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 784b8ca3a006eb051cd4eb81274042da, NAME => 'testInvalidSplitKey,,1675785336638.784b8ca3a006eb051cd4eb81274042da.', STARTKEY => '', ENDKEY => ''}] 2023-02-07 15:55:39,186 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-07 15:55:39,186 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675785339186"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:39,187 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-07 15:55:39,196 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:39,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 55 msec 2023-02-07 15:55:39,244 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:39,245 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:39,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-07 15:55:39,247 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-07 15:55:39,266 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=368 (was 349) - Thread LEAK? -, OpenFileDescriptor=742 (was 655) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=213 (was 213), ProcessCount=174 (was 174), AvailableMemoryMB=3078 (was 3106) 2023-02-07 15:55:39,282 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=368, OpenFileDescriptor=742, MaxFileDescriptor=60000, SystemLoadAverage=213, ProcessCount=174, AvailableMemoryMB=3078 2023-02-07 15:55:39,283 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:55:39,283 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:39,283 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:39,290 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 15:55:39,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 15:55:39,291 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:39,293 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 15:55:39,294 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 15:55:39,295 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:39,296 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00 empty. 2023-02-07 15:55:39,296 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:39,296 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-07 15:55:39,311 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-07 15:55:39,313 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0f24d22640b51f10471c5d20c64f2a00, NAME => 'testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp 2023-02-07 15:55:39,323 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:39,323 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 0f24d22640b51f10471c5d20c64f2a00, disabling compactions & flushes 2023-02-07 15:55:39,323 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:39,323 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:39,323 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. after waiting 0 ms 2023-02-07 15:55:39,323 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:39,323 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:39,323 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 0f24d22640b51f10471c5d20c64f2a00: Waiting for close lock at 1675785339323Disabling compacts and flushes for region at 1675785339323Disabling writes for close at 1675785339323Writing region close event to WAL at 1675785339323Closed at 1675785339323 2023-02-07 15:55:39,326 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 15:55:39,327 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785339327"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785339327"}]},"ts":"1675785339327"} 2023-02-07 15:55:39,329 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 15:55:39,330 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 15:55:39,330 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785339330"}]},"ts":"1675785339330"} 2023-02-07 15:55:39,331 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-07 15:55:39,353 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:55:39,355 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:55:39,355 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:55:39,355 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:55:39,355 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 15:55:39,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, ASSIGN}] 2023-02-07 15:55:39,358 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, ASSIGN 2023-02-07 15:55:39,359 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,37399,1675785236898; forceNewPlan=false, retain=false 2023-02-07 15:55:39,509 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:55:39,510 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=0f24d22640b51f10471c5d20c64f2a00, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,511 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785339510"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785339510"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785339510"}]},"ts":"1675785339510"} 2023-02-07 15:55:39,513 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898}] 2023-02-07 15:55:39,627 ERROR [BP-1532654085-136.243.104.168-1675785230655 heartbeating to localhost.localdomain/127.0.0.1:43555] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1532654085-136.243.104.168-1675785230655 (Datanode Uuid 8f3909c3-b4fb-4933-8c65-751b48eaa405) service to localhost.localdomain/127.0.0.1:43555 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 15:55:39,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:39,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0f24d22640b51f10471c5d20c64f2a00, NAME => 'testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:55:39,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:55:39,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:39,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:39,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:39,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:39,675 INFO [StoreOpener-0f24d22640b51f10471c5d20c64f2a00-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:39,678 DEBUG [StoreOpener-0f24d22640b51f10471c5d20c64f2a00-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1 2023-02-07 15:55:39,678 DEBUG [StoreOpener-0f24d22640b51f10471c5d20c64f2a00-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1 2023-02-07 15:55:39,678 INFO [StoreOpener-0f24d22640b51f10471c5d20c64f2a00-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f24d22640b51f10471c5d20c64f2a00 columnFamilyName cf1 2023-02-07 15:55:39,679 INFO [StoreOpener-0f24d22640b51f10471c5d20c64f2a00-1] regionserver.HStore(310): Store=0f24d22640b51f10471c5d20c64f2a00/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:39,680 INFO [StoreOpener-0f24d22640b51f10471c5d20c64f2a00-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:39,682 DEBUG [StoreOpener-0f24d22640b51f10471c5d20c64f2a00-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2 2023-02-07 15:55:39,682 DEBUG [StoreOpener-0f24d22640b51f10471c5d20c64f2a00-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2 2023-02-07 15:55:39,682 INFO [StoreOpener-0f24d22640b51f10471c5d20c64f2a00-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f24d22640b51f10471c5d20c64f2a00 columnFamilyName cf2 2023-02-07 15:55:39,683 INFO [StoreOpener-0f24d22640b51f10471c5d20c64f2a00-1] regionserver.HStore(310): Store=0f24d22640b51f10471c5d20c64f2a00/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:39,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:39,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:39,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:55:39,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:39,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:55:39,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 0f24d22640b51f10471c5d20c64f2a00; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59242451, jitterRate=-0.11721868813037872}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:55:39,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 0f24d22640b51f10471c5d20c64f2a00: Running coprocessor pre-open hook at 1675785339674Writing region info on filesystem at 1675785339674Initializing all the Stores at 1675785339675 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785339675Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785339675Cleaning up temporary data from old regions at 1675785339687 (+12 ms)Cleaning up detritus from prior splits at 1675785339687Running coprocessor post-open hooks at 1675785339696 (+9 ms)Region opened successfully at 1675785339696 2023-02-07 15:55:39,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00., pid=111, masterSystemTime=1675785339665 2023-02-07 15:55:39,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:39,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:39,700 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=0f24d22640b51f10471c5d20c64f2a00, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,700 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785339700"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785339700"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785339700"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785339700"}]},"ts":"1675785339700"} 2023-02-07 15:55:39,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-07 15:55:39,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 in 189 msec 2023-02-07 15:55:39,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-07 15:55:39,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, ASSIGN in 349 msec 2023-02-07 15:55:39,709 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 15:55:39,709 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785339709"}]},"ts":"1675785339709"} 2023-02-07 15:55:39,710 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-07 15:55:39,723 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 15:55:39,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 432 msec 2023-02-07 15:55:39,804 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:39,804 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:39,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:39,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-07 15:55:39,807 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:39,807 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:39,807 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-07 15:55:39,807 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 15:55:39,807 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,807 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,816 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,816 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:39,816 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:39,816 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:39,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:39,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:39,816 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,816 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:39,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:39,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:39,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:39,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:39,817 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:39,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:39,818 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:39,818 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:39,818 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. started... 2023-02-07 15:55:39,819 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f24d22640b51f10471c5d20c64f2a00 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 15:55:39,827 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,830 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/1e962a23ac2d4a6a918e473a97bd7897 2023-02-07 15:55:39,837 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,837 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:39,837 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,837 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,838 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:39,845 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/d52762de014247ab93fb1632b213da2a 2023-02-07 15:55:39,848 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/1e962a23ac2d4a6a918e473a97bd7897 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897 2023-02-07 15:55:39,852 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:55:39,852 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/d52762de014247ab93fb1632b213da2a as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a 2023-02-07 15:55:39,856 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:55:39,856 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 0f24d22640b51f10471c5d20c64f2a00 in 37ms, sequenceid=6, compaction requested=false 2023-02-07 15:55:39,857 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:39,857 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f24d22640b51f10471c5d20c64f2a00: 2023-02-07 15:55:39,857 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:39,857 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:39,857 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:39,857 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,857 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:39,857 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:39,869 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,869 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:39,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:39,869 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,869 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:39,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:39,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:39,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:39,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:39,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:39,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:39,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-07 15:55:39,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ce98ab5[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:39,872 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 15:55:39,872 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,879 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,879 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,879 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,880 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:39,880 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:39,880 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:39,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,880 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,880 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:39,880 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:39,880 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:39,881 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:39,881 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:39,881 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:39,881 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:39,890 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:39,890 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:39,890 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:39,900 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:39,900 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,900 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:39,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:39,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:39,900 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:39,901 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:39,900 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:39,901 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:39,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:39,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:39,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:39,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:39,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:39,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:39,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:39,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:39,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:39,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,37399,1675785236898': 2023-02-07 15:55:39,904 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 15:55:39,905 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-07 15:55:39,905 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:39,905 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:39,905 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-07 15:55:39,905 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:39,911 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,911 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,911 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:39,911 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,911 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:39,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,911 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:39,911 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:39,911 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:39,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:39,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:39,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:39,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:39,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:39,912 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:39,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:39,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:39,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:39,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,912 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:39,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:39,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:39,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:39,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:39,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,914 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:39,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:39,915 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:39,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:39,915 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:39,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:39,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,921 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,921 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:39,921 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:39,921 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,921 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:39,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:39,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:39,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:39,922 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:39,922 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:39,922 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:39,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:39,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:39,921 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:39,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:39,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:39,921 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:39,922 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:39,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:39,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:39,922 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:39,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:39,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:39,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:39,923 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,923 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:39,923 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:39,923 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,923 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:39,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:39,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:39,923 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:39,924 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:39,924 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,023 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:40,024 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:40,024 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:40,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:40,043 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:40,043 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:40,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:40,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:40,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-07 15:55:40,045 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:40,045 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,045 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-07 15:55:40,045 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 15:55:40,045 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,045 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,058 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,058 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,058 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,058 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,059 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,059 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:40,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,060 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:40,060 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:40,060 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:40,060 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,060 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:40,060 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:40,060 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,060 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:40,060 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:40,061 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:40,061 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. started... 2023-02-07 15:55:40,061 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f24d22640b51f10471c5d20c64f2a00 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:40,072 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/d15ceae042244d1587dfd9c4fb2743eb 2023-02-07 15:55:40,084 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/084b046260fc45a28fc2617ff0951865 2023-02-07 15:55:40,088 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/d15ceae042244d1587dfd9c4fb2743eb as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb 2023-02-07 15:55:40,094 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:55:40,095 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/084b046260fc45a28fc2617ff0951865 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865 2023-02-07 15:55:40,100 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:55:40,102 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f24d22640b51f10471c5d20c64f2a00 in 41ms, sequenceid=19, compaction requested=false 2023-02-07 15:55:40,102 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f24d22640b51f10471c5d20c64f2a00: 2023-02-07 15:55:40,102 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:40,102 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:40,102 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:40,102 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,102 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:40,102 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:40,125 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,125 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,142 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,142 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:40,142 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,142 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:40,142 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,142 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:40,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:40,143 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,143 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:40,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:40,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:40,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:40,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-07 15:55:40,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6838862b[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:40,145 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 15:55:40,145 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,153 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,153 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,153 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,153 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:40,153 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:40,153 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:40,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,153 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:40,153 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:40,153 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,154 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:40,154 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:40,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,154 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:40,154 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:40,154 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:40,163 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:40,163 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,164 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:40,174 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:40,174 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,174 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:40,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:40,174 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:40,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:40,184 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:40,184 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,185 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:40,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:40,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:40,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,37399,1675785236898': 2023-02-07 15:55:40,190 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 15:55:40,190 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-07 15:55:40,190 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:40,190 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:40,190 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,190 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:40,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:40,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:40,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:40,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,205 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,206 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:40,207 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:40,210 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,211 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,212 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:40,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:40,227 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,228 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:40,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,227 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,228 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:40,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,229 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,229 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,229 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,229 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,230 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,230 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,230 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,230 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,329 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:40,329 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:40,329 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:40,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:40,346 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:40,346 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:40,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:40,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:40,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-07 15:55:40,350 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:40,350 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,350 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-07 15:55:40,350 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 15:55:40,351 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,351 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,364 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,364 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,364 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,364 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,364 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,365 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,366 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,366 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,367 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:40,367 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,367 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:40,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:40,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:40,368 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,368 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:40,368 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:40,369 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,368 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,369 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:40,369 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:40,369 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,368 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,369 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:40,369 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:40,369 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:40,369 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:40,369 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:40,369 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:40,370 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:40,369 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:40,369 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:40,370 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. started... 2023-02-07 15:55:40,370 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f24d22640b51f10471c5d20c64f2a00 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:40,383 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,383 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,383 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,383 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,383 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:40,383 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:40,384 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/5e46ea137223454d916714ad7a522357 2023-02-07 15:55:40,398 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/f0cc35d41bc74940837ec44ef16760a9 2023-02-07 15:55:40,402 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/5e46ea137223454d916714ad7a522357 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357 2023-02-07 15:55:40,405 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:55:40,405 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/f0cc35d41bc74940837ec44ef16760a9 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9 2023-02-07 15:55:40,408 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:55:40,409 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f24d22640b51f10471c5d20c64f2a00 in 39ms, sequenceid=32, compaction requested=true 2023-02-07 15:55:40,409 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f24d22640b51f10471c5d20c64f2a00: 2023-02-07 15:55:40,409 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:40,409 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:40,409 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:40,409 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,409 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:40,409 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:40,416 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,416 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:40,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:40,416 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,416 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:40,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:40,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:40,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:40,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-07 15:55:40,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56712524[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:40,418 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 15:55:40,418 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,427 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,427 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,427 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,427 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:40,427 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:40,427 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:40,427 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,427 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:40,427 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:40,427 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:40,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,428 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:40,428 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:40,428 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:40,428 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:40,437 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:40,437 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,437 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:40,474 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,474 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:40,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:40,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:40,474 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,474 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:40,475 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,475 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:40,475 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:40,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:40,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:40,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:40,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,37399,1675785236898': 2023-02-07 15:55:40,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 15:55:40,485 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-07 15:55:40,485 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:40,485 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:40,485 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,485 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:40,501 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,501 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,501 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,501 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,501 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,501 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,502 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:40,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,502 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:40,503 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:40,505 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:40,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,508 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,508 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:40,509 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,521 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,521 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,522 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,521 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,522 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,522 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,522 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:40,522 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:40,522 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,522 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,522 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,522 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,523 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,523 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,523 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,523 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,523 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,522 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,523 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,522 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:40,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,522 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,524 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:40,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,624 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:40,624 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:40,625 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:40,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:40,644 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:40,644 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:40,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:40,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:40,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-07 15:55:40,646 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:40,647 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,647 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-07 15:55:40,647 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 15:55:40,647 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,647 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,658 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,658 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,658 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,658 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,659 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,660 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:40,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,660 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:40,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:40,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,661 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:40,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:40,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,662 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:40,662 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:40,662 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:40,662 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:40,662 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,662 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,662 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:40,663 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,662 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:40,663 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:40,663 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:40,663 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,663 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,663 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:40,663 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. started... 2023-02-07 15:55:40,663 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:40,663 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:40,663 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:40,663 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:40,664 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:40,664 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f24d22640b51f10471c5d20c64f2a00 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:40,663 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:40,664 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,664 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:40,664 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:40,664 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:40,669 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,679 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,679 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,679 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:40,679 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,680 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:40,687 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/caf908aec3d24f9497871245c2a466e1 2023-02-07 15:55:40,699 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/5014a511e4804a50b6a77dff3215a44a 2023-02-07 15:55:40,703 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/caf908aec3d24f9497871245c2a466e1 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1 2023-02-07 15:55:40,706 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:55:40,707 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/5014a511e4804a50b6a77dff3215a44a as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a 2023-02-07 15:55:40,710 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:55:40,711 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f24d22640b51f10471c5d20c64f2a00 in 48ms, sequenceid=45, compaction requested=true 2023-02-07 15:55:40,711 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f24d22640b51f10471c5d20c64f2a00: 2023-02-07 15:55:40,711 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:40,711 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:40,711 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:40,711 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,711 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:40,711 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:40,721 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,721 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:40,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:40,722 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,722 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:40,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:40,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:40,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:40,723 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-07 15:55:40,724 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a7a29c9[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:40,724 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 15:55:40,724 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,732 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,732 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,732 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,732 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,732 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:40,732 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:40,732 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:40,732 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:40,732 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:40,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,732 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:40,733 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:40,733 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:40,733 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:40,733 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:40,742 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:40,742 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,742 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:40,753 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:40,753 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,753 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:40,753 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:40,753 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,753 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:40,753 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:40,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:40,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:40,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:40,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:40,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,37399,1675785236898': 2023-02-07 15:55:40,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 15:55:40,757 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-07 15:55:40,757 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:40,757 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:40,757 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,757 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:40,763 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,763 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,763 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,763 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,763 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,763 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:40,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:40,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,764 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,764 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:40,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,765 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,765 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:40,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,766 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:40,766 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,766 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,774 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,774 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,774 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,774 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,774 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,774 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,774 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,774 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,774 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,774 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:40,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:40,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,774 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:40,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,775 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,775 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:40,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,775 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:40,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:40,775 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:40,775 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,775 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,775 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,775 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,775 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,875 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:40,876 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:40,876 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:40,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:40,898 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:40,898 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:40,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:40,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:40,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-07 15:55:40,901 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:40,901 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,901 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-07 15:55:40,901 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 15:55:40,902 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,902 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,911 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,911 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,911 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,911 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:40,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,911 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:40,911 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,912 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:40,912 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,912 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,912 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:40,912 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:40,912 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,912 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,912 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:40,913 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:40,913 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:40,912 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:40,913 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:40,913 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:40,913 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:40,913 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:40,913 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:40,913 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:40,913 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:40,913 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:40,913 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:40,913 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:40,913 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. started... 2023-02-07 15:55:40,913 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f24d22640b51f10471c5d20c64f2a00 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:40,929 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/ae9d35e519d74f0894c2de4caa51300f 2023-02-07 15:55:40,940 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/8e2147f6cb7a4cc7a55e76ea182e93af 2023-02-07 15:55:40,943 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/ae9d35e519d74f0894c2de4caa51300f as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f 2023-02-07 15:55:40,947 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:55:40,947 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/8e2147f6cb7a4cc7a55e76ea182e93af as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af 2023-02-07 15:55:40,949 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,951 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:55:40,952 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f24d22640b51f10471c5d20c64f2a00 in 39ms, sequenceid=58, compaction requested=true 2023-02-07 15:55:40,952 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f24d22640b51f10471c5d20c64f2a00: 2023-02-07 15:55:40,952 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:40,952 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:40,952 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:40,952 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,952 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:40,952 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:40,963 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,964 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,964 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:40,974 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,974 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,974 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:40,974 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:40,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:40,975 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,975 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:40,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:40,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:40,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:40,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:40,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:40,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:40,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-07 15:55:40,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a46b411[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:40,979 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 15:55:40,979 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,985 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,985 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,985 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,985 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:40,985 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:40,986 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:40,986 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:40,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,985 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:40,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:40,986 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:40,986 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:40,986 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:40,986 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:40,986 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:40,986 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:40,995 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:40,995 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:40,995 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:41,006 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:41,006 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:41,006 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:41,006 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,006 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:41,006 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:41,007 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:41,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:41,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:41,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:41,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:41,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:41,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:41,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:41,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,37399,1675785236898': 2023-02-07 15:55:41,013 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 15:55:41,013 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-07 15:55:41,013 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:41,013 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:41,013 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-07 15:55:41,013 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:41,025 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,025 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:41,025 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,025 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:41,025 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,025 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:41,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,025 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,026 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:41,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:41,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:41,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:41,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:41,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:41,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:41,027 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:41,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:41,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,028 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:41,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:41,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:41,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,030 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:41,031 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:41,031 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,046 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:41,046 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:41,046 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:41,046 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:41,046 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:41,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:41,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:41,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:41,046 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,047 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,047 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:41,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:41,047 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:41,047 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:41,046 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:41,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:41,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:41,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:41,046 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:41,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:41,047 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,047 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:41,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:41,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:41,048 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:41,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:41,048 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,048 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,048 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,048 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,048 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,148 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:41,148 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:41,149 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:41,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:41,171 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:41,171 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:41,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:41,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:41,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-07 15:55:41,173 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:41,173 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:41,174 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-07 15:55:41,174 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 15:55:41,174 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,174 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,185 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,185 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:41,185 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:41,185 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:41,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:41,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:41,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:41,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:41,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:41,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:41,185 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,186 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:41,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:41,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:41,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:41,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:41,187 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:41,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:41,187 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:41,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,187 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:41,187 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:41,187 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:41,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 15:55:41,188 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:41,188 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:41,188 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:41,188 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 15:55:41,187 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:41,188 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 15:55:41,188 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:41,188 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:41,188 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 15:55:41,188 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:41,188 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:41,188 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 15:55:41,188 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:41,188 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:41,188 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:41,189 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:41,189 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:41,189 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. started... 2023-02-07 15:55:41,189 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f24d22640b51f10471c5d20c64f2a00 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:41,195 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,200 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/bae7176e038244d5a366c0b4f9a8d2f4 2023-02-07 15:55:41,205 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,206 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:41,205 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,206 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,206 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:41,218 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/84abd4ec29984e5790af488048f874c5 2023-02-07 15:55:41,224 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/bae7176e038244d5a366c0b4f9a8d2f4 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4 2023-02-07 15:55:41,228 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:55:41,228 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/84abd4ec29984e5790af488048f874c5 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5 2023-02-07 15:55:41,234 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:55:41,235 INFO [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f24d22640b51f10471c5d20c64f2a00 in 46ms, sequenceid=71, compaction requested=true 2023-02-07 15:55:41,235 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f24d22640b51f10471c5d20c64f2a00: 2023-02-07 15:55:41,235 DEBUG [rs(jenkins-hbase12.apache.org,37399,1675785236898)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:41,235 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:41,235 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:41,235 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,235 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 15:55:41,235 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 15:55:41,248 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,248 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:41,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:41,248 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,248 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:41,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:41,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:41,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:41,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:41,250 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-07 15:55:41,250 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@366370ac[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:41,250 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 15:55:41,250 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,258 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,258 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,258 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,258 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,259 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:41,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,258 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,259 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:41,259 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:41,259 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:41,259 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:41,259 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 15:55:41,259 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 15:55:41,259 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:41,259 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:41,279 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:41,279 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:41,279 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:41,279 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:41,280 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:41,280 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:41,318 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 15:55:41,318 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:41,319 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 15:55:41,318 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:41,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:41,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:41,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,37399,1675785236898': 2023-02-07 15:55:41,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 15:55:41,322 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-07 15:55:41,322 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:41,322 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:41,322 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-07 15:55:41,322 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:41,342 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,342 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,342 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,342 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,343 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,343 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:41,342 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:41,343 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:41,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:41,343 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,344 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,345 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:41,345 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 15:55:41,345 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,353 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:41,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:41,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:41,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:41,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:41,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:41,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:41,353 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:41,354 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:41,354 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:41,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:41,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:41,354 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:41,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:41,354 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:41,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:41,354 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:41,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:41,354 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 15:55:41,454 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:41,454 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:41,454 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:41,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:41,712 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-07 15:55:41,713 DEBUG [Listener at localhost.localdomain/41965] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:41,715 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:41,716 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:41,716 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:41,716 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:41,717 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:55:41,718 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:41,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:55:41,718 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:55:41,965 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:41,965 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:55:41,965 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:55:41,965 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:55:41,966 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[IPC Client (686864534) connection to localhost.localdomain/127.0.0.1:43555 from jenkins,5,PEWorkerGroup] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-07 15:55:41,966 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:55:41,966 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:55:41,966 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:55:41,966 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:55:41,967 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:55:41,968 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-07 15:55:41,968 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:55:41,969 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:55:41,969 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:55:41,969 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:55:41,969 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:41,970 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:55:41,970 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:41,970 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:41,970 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:41,971 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:55:41,971 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:41,971 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:41,971 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:41,971 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:55:41,971 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:55:41,972 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:41,972 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:41,972 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:41,972 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:55:41,972 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:41,973 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:41,973 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:41,973 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:55:41,973 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:41,973 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:41,974 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:41,974 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:41,975 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:41,975 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:41,975 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:41,976 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:41,976 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:41,977 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:41,977 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:41,977 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:41,977 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:55:41,977 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:41,977 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:41,978 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:41,978 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-07 15:55:41,978 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:55:41,979 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:55:41,979 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785371979 2023-02-07 15:55:41,980 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:55:41,980 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:41,982 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 15:55:41,984 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:55:41,984 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=0f24d22640b51f10471c5d20c64f2a00, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=2 2023-02-07 15:55:41,984 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:55:41,984 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785401984 2023-02-07 15:55:42,003 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785462003 2023-02-07 15:55:42,003 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 22 msec 2023-02-07 15:55:42,003 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:55:42,005 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:55:42,005 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:55:42,005 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:55:42,253 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,253 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:55:42,254 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:55:42,254 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:55:42,256 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:55:42,256 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:55:42,256 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:55:42,257 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:55:42,257 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:55:42,259 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 15:55:42,262 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-07 15:55:42,263 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:55:42,263 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:55:42,263 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:55:42,263 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:55:42,264 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:42,264 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:55:42,264 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:42,264 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:42,265 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:42,265 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:55:42,265 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:42,265 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:42,266 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:42,266 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:55:42,266 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:55:42,266 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:42,266 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:42,267 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:42,267 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:55:42,267 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:42,267 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:42,268 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:42,268 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:55:42,268 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:42,268 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:42,269 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:42,269 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:42,269 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:42,269 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:42,269 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:42,269 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:42,269 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:42,269 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:42,270 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:42,270 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,270 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:55:42,270 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,270 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,270 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,270 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-07 15:55:42,271 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:55:42,271 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:55:42,272 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785372272 2023-02-07 15:55:42,272 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:55:42,273 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,275 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:55:42,275 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=0f24d22640b51f10471c5d20c64f2a00, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=2 2023-02-07 15:55:42,275 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:55:42,275 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785402275 2023-02-07 15:55:42,275 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785462275 2023-02-07 15:55:42,275 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:55:42,275 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:55:42,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN}] 2023-02-07 15:55:42,277 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:55:42,277 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:55:42,277 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:55:42,525 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,525 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:55:42,526 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:55:42,526 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:55:42,526 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:55:42,527 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:55:42,527 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:55:42,527 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:55:42,527 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:55:42,529 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 15:55:42,533 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-07 15:55:42,533 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:55:42,534 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:55:42,534 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:55:42,534 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:55:42,534 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:42,535 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:55:42,535 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:42,535 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:42,535 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:42,536 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:55:42,536 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:42,536 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:42,536 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:42,537 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:55:42,537 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:55:42,537 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:42,537 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:42,537 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:42,537 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:55:42,537 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:42,537 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:42,538 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:42,538 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:55:42,538 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:42,538 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:42,538 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:42,538 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:42,539 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:42,539 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:42,539 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:42,539 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:42,539 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:42,539 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:42,539 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:42,539 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,540 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:55:42,540 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,540 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,540 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,540 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-07 15:55:42,540 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:55:42,541 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:55:42,541 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785372541 2023-02-07 15:55:42,542 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:55:42,542 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:42,544 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:55:42,545 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=0f24d22640b51f10471c5d20c64f2a00, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=2 2023-02-07 15:55:42,545 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:55:42,545 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785402545 2023-02-07 15:55:42,545 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785462545 2023-02-07 15:55:42,545 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:55:42,545 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN}] 2023-02-07 15:55:42,545 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:55:42,548 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN 2023-02-07 15:55:42,548 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=0f24d22640b51f10471c5d20c64f2a00, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:42,548 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785342548"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785342548"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785342548"}]},"ts":"1675785342548"} 2023-02-07 15:55:42,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898}] 2023-02-07 15:55:42,626 ERROR [BP-1532654085-136.243.104.168-1675785230655 heartbeating to localhost.localdomain/127.0.0.1:43555] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1532654085-136.243.104.168-1675785230655 (Datanode Uuid 7f437b77-6c8e-4bcb-8556-91fa3c36506b) service to localhost.localdomain/127.0.0.1:43555 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 15:55:42,702 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:42,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 0f24d22640b51f10471c5d20c64f2a00, disabling compactions & flushes 2023-02-07 15:55:42,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:42,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:42,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. after waiting 0 ms 2023-02-07 15:55:42,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:42,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 0f24d22640b51f10471c5d20c64f2a00 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 15:55:43,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/36a22297520746069727bf9f12d7a581 2023-02-07 15:55:43,538 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/4787d0f13eee4eac82743f6c164f999a 2023-02-07 15:55:43,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf1/36a22297520746069727bf9f12d7a581 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581 2023-02-07 15:55:43,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:55:43,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/.tmp/cf2/4787d0f13eee4eac82743f6c164f999a as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a 2023-02-07 15:55:43,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:55:43,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 0f24d22640b51f10471c5d20c64f2a00 in 859ms, sequenceid=82, compaction requested=true 2023-02-07 15:55:43,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 15:55:43,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:55:43,568 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. 2023-02-07 15:55:43,568 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 0f24d22640b51f10471c5d20c64f2a00: Waiting for close lock at 1675785342702Running coprocessor pre-close hooks at 1675785342702Disabling compacts and flushes for region at 1675785342702Disabling writes for close at 1675785342703 (+1 ms)Obtaining lock to block concurrent updates at 1675785342703Preparing flush snapshotting stores in 0f24d22640b51f10471c5d20c64f2a00 at 1675785342703Finished memstore snapshotting testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675785342704 (+1 ms)Flushing stores of testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. at 1675785342705 (+1 ms)Flushing 0f24d22640b51f10471c5d20c64f2a00/cf1: creating writer at 1675785342705Flushing 0f24d22640b51f10471c5d20c64f2a00/cf1: appending metadata at 1675785342707 (+2 ms)Flushing 0f24d22640b51f10471c5d20c64f2a00/cf1: closing flushed file at 1675785342707Flushing 0f24d22640b51f10471c5d20c64f2a00/cf2: creating writer at 1675785343123 (+416 ms)Flushing 0f24d22640b51f10471c5d20c64f2a00/cf2: appending metadata at 1675785343124 (+1 ms)Flushing 0f24d22640b51f10471c5d20c64f2a00/cf2: closing flushed file at 1675785343125 (+1 ms)Flushing 0f24d22640b51f10471c5d20c64f2a00/cf1: reopening flushed file at 1675785343551 (+426 ms)Flushing 0f24d22640b51f10471c5d20c64f2a00/cf2: reopening flushed file at 1675785343557 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 0f24d22640b51f10471c5d20c64f2a00 in 859ms, sequenceid=82, compaction requested=true at 1675785343563 (+6 ms)Writing region close event to WAL at 1675785343564 (+1 ms)Running coprocessor post-close hooks at 1675785343567 (+3 ms)Closed at 1675785343568 (+1 ms) 2023-02-07 15:55:43,570 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,571 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=0f24d22640b51f10471c5d20c64f2a00, regionState=CLOSED 2023-02-07 15:55:43,571 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785343571"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785343571"}]},"ts":"1675785343571"} 2023-02-07 15:55:43,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-07 15:55:43,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 in 1.0230 sec 2023-02-07 15:55:43,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-07 15:55:43,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN in 1.0290 sec 2023-02-07 15:55:43,577 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:55:43,577 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:43,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:55:43,577 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:55:43,796 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:43,796 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:55:43,796 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:55:43,796 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:55:43,796 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:55:43,796 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:55:43,796 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:55:43,796 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:55:43,797 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:55:43,797 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:55:43,799 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 15:55:43,799 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:55:43,799 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:55:43,799 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:55:43,800 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:55:43,800 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:55:43,800 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:55:43,800 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:43,800 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:55:43,801 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:43,801 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:43,801 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:43,801 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:55:43,801 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:43,801 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:43,802 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:43,802 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:55:43,802 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:55:43,802 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:43,802 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:43,803 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:43,803 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:55:43,803 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:43,803 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:43,804 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:43,804 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:55:43,804 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:43,804 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:43,805 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:43,805 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:43,805 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:43,805 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:43,806 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:43,806 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:43,806 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:43,806 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:43,806 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:43,807 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:43,807 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN 2023-02-07 15:55:43,807 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:43,807 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:55:43,807 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:43,808 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:43,808 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:43,808 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:43,808 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:43,808 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-07 15:55:43,809 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:55:43,809 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:55:43,809 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785373809 2023-02-07 15:55:43,810 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:55:43,810 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:43,812 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:55:43,813 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=0f24d22640b51f10471c5d20c64f2a00, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=2 2023-02-07 15:55:43,813 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:55:43,813 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785403813 2023-02-07 15:55:43,813 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785463813 2023-02-07 15:55:43,813 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:55:43,815 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:55:43,824 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=0f24d22640b51f10471c5d20c64f2a00, threads=14 2023-02-07 15:55:43,824 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,825 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,825 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,825 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,825 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,825 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,825 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,826 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,827 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,827 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,827 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,828 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,828 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:43,828 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,021 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,022 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,022 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,045 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,045 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,052 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,057 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,061 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,066 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,074 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,074 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,074 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,075 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,439 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:44,439 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 0f24d22640b51f10471c5d20c64f2a00 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 15:55:44,895 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:55:44,895 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:44,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:55:44,895 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:55:45,063 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,064 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:55:45,064 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:55:45,064 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:55:45,066 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:55:45,067 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:55:45,067 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:55:45,067 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:55:45,068 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:55:45,069 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 15:55:45,073 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-07 15:55:45,073 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:55:45,074 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:55:45,074 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:55:45,075 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:55:45,076 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:55:45,076 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:55:45,077 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:45,077 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:55:45,077 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:45,077 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:45,077 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:45,077 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:55:45,077 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:45,077 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:45,078 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:45,078 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:55:45,078 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:55:45,078 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:45,078 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:45,078 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:45,078 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:55:45,078 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:45,079 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:45,079 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:45,079 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:55:45,079 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:45,079 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:45,079 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:45,079 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:45,079 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:45,079 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:45,080 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:45,080 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:45,080 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:45,080 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:45,080 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:45,080 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,080 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN 2023-02-07 15:55:45,080 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:45,080 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:55:45,080 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,081 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,081 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,081 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:45,081 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:45,081 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-07 15:55:45,082 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:55:45,082 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:55:45,082 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785375082 2023-02-07 15:55:45,083 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,083 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:55:45,086 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:55:45,086 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=0f24d22640b51f10471c5d20c64f2a00, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=2 2023-02-07 15:55:45,086 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:55:45,086 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785405086 2023-02-07 15:55:45,086 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785465086 2023-02-07 15:55:45,086 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:55:45,093 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=0f24d22640b51f10471c5d20c64f2a00, threads=14 2023-02-07 15:55:45,094 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,094 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,094 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,094 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,094 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,094 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,094 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,095 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,095 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,095 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,095 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,095 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,095 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,095 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,151 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,154 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,159 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,160 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,162 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,169 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,169 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,171 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,172 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,173 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,174 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,175 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,571 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4 for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,572 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a for region: 0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:45,572 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 0f24d22640b51f10471c5d20c64f2a00 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 15:55:45,615 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:55:45,619 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 15:55:45,621 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 15:55:45,621 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:55:45,621 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:55:45,621 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:55:45,626 ERROR [BP-1532654085-136.243.104.168-1675785230655 heartbeating to localhost.localdomain/127.0.0.1:43555] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1532654085-136.243.104.168-1675785230655 (Datanode Uuid bb8a498b-c978-4fc3-b7cd-17da28adf753) service to localhost.localdomain/127.0.0.1:43555 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 15:55:45,837 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,837 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:55:45,837 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:55:45,837 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:55:45,837 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:55:45,837 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:55:45,837 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:55:45,838 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:55:45,838 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:55:45,838 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:55:45,840 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-07 15:55:45,840 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:55:45,840 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:55:45,841 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:55:45,841 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:55:45,841 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:55:45,842 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:55:45,842 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:45,842 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:55:45,842 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:45,842 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:45,843 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:45,843 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:55:45,843 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:45,843 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:45,844 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:45,844 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:55:45,844 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:55:45,845 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:45,845 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:45,845 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:45,845 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:55:45,845 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:45,845 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:45,845 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:45,846 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:55:45,846 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:45,846 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:45,846 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:45,846 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:45,846 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:45,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:45,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:45,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:45,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:45,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:45,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:45,847 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,848 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN 2023-02-07 15:55:45,848 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:45,848 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:55:45,848 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,848 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,848 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,848 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:45,848 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:45,848 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-07 15:55:45,849 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:55:45,849 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:55:45,850 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785375850 2023-02-07 15:55:45,850 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:55:45,851 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,853 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:55:45,853 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=0f24d22640b51f10471c5d20c64f2a00, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=2 2023-02-07 15:55:45,853 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:55:45,853 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785405853 2023-02-07 15:55:45,854 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785465854 2023-02-07 15:55:45,854 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:55:45,856 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:55:45,858 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:55:45,858 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:45,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:55:45,858 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:55:46,104 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,104 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:55:46,104 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:55:46,104 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:55:46,105 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:55:46,105 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:55:46,105 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:55:46,105 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:55:46,106 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:55:46,107 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 15:55:46,111 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-07 15:55:46,112 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:55:46,112 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:55:46,112 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:55:46,113 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:55:46,113 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:55:46,113 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:55:46,113 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:46,113 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:55:46,113 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:46,114 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:46,114 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:46,114 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:55:46,114 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:46,114 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:46,115 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:46,115 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:55:46,115 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:55:46,115 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:46,115 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:46,115 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:46,116 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:55:46,116 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:46,116 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:46,116 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:46,116 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:55:46,116 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:46,116 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:46,117 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,117 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,117 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:46,117 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,118 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,118 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:46,118 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:46,118 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:46,118 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:46,119 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,119 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN 2023-02-07 15:55:46,119 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:46,119 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:55:46,119 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,119 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,119 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,119 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:46,119 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:46,119 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-07 15:55:46,120 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:55:46,120 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:55:46,121 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785376120 2023-02-07 15:55:46,122 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,122 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:55:46,124 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:55:46,124 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=0f24d22640b51f10471c5d20c64f2a00, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=2 2023-02-07 15:55:46,124 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:55:46,124 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785406124 2023-02-07 15:55:46,124 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785466124 2023-02-07 15:55:46,125 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:55:46,125 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:55:46,127 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785346127"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675785346127"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675785346127"}]},"ts":"1675785346127"} 2023-02-07 15:55:46,127 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675785346127"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785346127"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785346127"}]},"ts":"1675785346127"} 2023-02-07 15:55:46,127 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675785346127"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785346127"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785346127"}]},"ts":"1675785346127"} 2023-02-07 15:55:46,129 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:55:46,129 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:55:46,129 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:55:46,375 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,375 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:55:46,375 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:55:46,375 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:55:46,377 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:55:46,377 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:55:46,377 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:55:46,377 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:55:46,377 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:55:46,378 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:55:46,379 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-07 15:55:46,379 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:55:46,380 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:55:46,380 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:55:46,380 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:55:46,380 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:55:46,380 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:55:46,381 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:46,381 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:55:46,381 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:46,381 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:46,381 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:46,382 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:55:46,382 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:46,382 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:46,382 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:46,383 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:55:46,383 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:55:46,383 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:46,383 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:46,383 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:46,384 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:55:46,384 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:46,384 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:46,384 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:46,385 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:55:46,385 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:46,385 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:46,385 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,386 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,386 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:46,387 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,387 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,387 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:46,387 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:46,388 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:46,388 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:46,388 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,388 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN 2023-02-07 15:55:46,389 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:46,389 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:55:46,389 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,389 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,389 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,389 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:46,389 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:46,389 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-07 15:55:46,390 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:55:46,390 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:55:46,392 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785376391 2023-02-07 15:55:46,392 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,392 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:55:46,396 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:55:46,397 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=0f24d22640b51f10471c5d20c64f2a00, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=2 2023-02-07 15:55:46,397 INFO [Listener at localhost.localdomain/41965] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:46,397 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=cfc33a4a114706f42376378024a7423f, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-07 15:55:46,397 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=48fead1a7a8db08778f1e1e72513eace, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-07 15:55:46,397 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:55:46,397 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785406397 2023-02-07 15:55:46,397 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785466397 2023-02-07 15:55:46,397 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-07 15:55:46,398 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675785346398"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675785346398"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675785346398"}]},"ts":"1675785346398"} 2023-02-07 15:55:46,398 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675785346398"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785346398"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785346398"}]},"ts":"1675785346398"} 2023-02-07 15:55:46,398 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675785346398"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785346398"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785346398"}]},"ts":"1675785346398"} 2023-02-07 15:55:46,399 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:55:46,401 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:55:46,401 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:55:46,401 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:55:46,648 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,648 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:55:46,648 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:55:46,648 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:55:46,648 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:55:46,649 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:55:46,649 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:55:46,649 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:55:46,649 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:55:46,650 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 15:55:46,652 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-07 15:55:46,652 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:55:46,652 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:55:46,653 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:55:46,653 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:55:46,653 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:55:46,653 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:55:46,654 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:46,654 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:55:46,654 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:46,655 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:46,655 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:46,655 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:55:46,656 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:46,656 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:46,656 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:46,657 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:55:46,657 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:55:46,657 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:46,657 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:46,657 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:46,657 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:55:46,657 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:46,657 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:46,658 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:46,658 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:55:46,658 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:46,658 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:46,658 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,658 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,658 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:46,658 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,659 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,659 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:46,659 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:46,659 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:46,659 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:46,659 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,659 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN 2023-02-07 15:55:46,659 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:46,659 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:55:46,660 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,660 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,660 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,660 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:46,660 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:46,660 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-07 15:55:46,661 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:55:46,661 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:55:46,661 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785376661 2023-02-07 15:55:46,662 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,662 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:55:46,664 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:55:46,665 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=0f24d22640b51f10471c5d20c64f2a00, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=2 2023-02-07 15:55:46,665 INFO [Listener at localhost.localdomain/41965] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:46,665 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=cfc33a4a114706f42376378024a7423f, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-07 15:55:46,665 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=48fead1a7a8db08778f1e1e72513eace, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-07 15:55:46,665 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:55:46,665 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785406665 2023-02-07 15:55:46,665 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785466665 2023-02-07 15:55:46,665 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 15:55:46,665 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:55:46,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=cfc33a4a114706f42376378024a7423f, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48fead1a7a8db08778f1e1e72513eace, ASSIGN}] 2023-02-07 15:55:46,667 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:55:46,667 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:55:46,667 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:55:46,916 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,916 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:55:46,916 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:55:46,916 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:55:46,917 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:55:46,917 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:55:46,917 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:55:46,917 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:55:46,918 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:55:46,919 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 15:55:46,921 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-07 15:55:46,921 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:55:46,921 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:55:46,922 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:55:46,922 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:55:46,922 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:55:46,922 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:55:46,922 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:46,923 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:55:46,923 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:46,923 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:46,923 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:46,923 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:55:46,923 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:46,923 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:46,924 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:46,924 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:55:46,924 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:55:46,924 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:46,924 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:46,924 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:46,925 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:55:46,925 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:46,925 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:46,925 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:46,925 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:55:46,925 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:46,925 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:46,926 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,926 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,926 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:46,926 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,926 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:46,926 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:46,927 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:46,927 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:46,927 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:46,927 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,927 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN 2023-02-07 15:55:46,927 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:46,927 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:55:46,928 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,928 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,928 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,928 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:46,928 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:46,928 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-07 15:55:46,929 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:55:46,929 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:55:46,930 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785376930 2023-02-07 15:55:46,930 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:55:46,931 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:46,933 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:55:46,933 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=0f24d22640b51f10471c5d20c64f2a00, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=2 2023-02-07 15:55:46,933 INFO [Listener at localhost.localdomain/41965] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:46,934 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=cfc33a4a114706f42376378024a7423f, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-07 15:55:46,934 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=48fead1a7a8db08778f1e1e72513eace, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-07 15:55:46,934 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:55:46,934 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785406934 2023-02-07 15:55:46,934 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785466934 2023-02-07 15:55:46,934 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-07 15:55:46,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=cfc33a4a114706f42376378024a7423f, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48fead1a7a8db08778f1e1e72513eace, ASSIGN}] 2023-02-07 15:55:46,934 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:55:46,936 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48fead1a7a8db08778f1e1e72513eace, ASSIGN 2023-02-07 15:55:46,936 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48fead1a7a8db08778f1e1e72513eace, ASSIGN; state=CLOSED, location=jenkins-hbase12.apache.org,37399,1675785236898; forceNewPlan=false, retain=false 2023-02-07 15:55:46,937 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=cfc33a4a114706f42376378024a7423f, ASSIGN 2023-02-07 15:55:46,937 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=cfc33a4a114706f42376378024a7423f, ASSIGN; state=CLOSED, location=jenkins-hbase12.apache.org,37399,1675785236898; forceNewPlan=false, retain=false 2023-02-07 15:55:47,087 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 15:55:47,089 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=cfc33a4a114706f42376378024a7423f, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:47,089 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675785347089"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785347089"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785347089"}]},"ts":"1675785347089"} 2023-02-07 15:55:47,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure cfc33a4a114706f42376378024a7423f, server=jenkins-hbase12.apache.org,37399,1675785236898}] 2023-02-07 15:55:47,096 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=48fead1a7a8db08778f1e1e72513eace, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:47,096 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675785347096"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785347096"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785347096"}]},"ts":"1675785347096"} 2023-02-07 15:55:47,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 48fead1a7a8db08778f1e1e72513eace, server=jenkins-hbase12.apache.org,37399,1675785236898}] 2023-02-07 15:55:47,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f. 2023-02-07 15:55:47,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cfc33a4a114706f42376378024a7423f, NAME => 'testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.', STARTKEY => '', ENDKEY => '41'} 2023-02-07 15:55:47,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:55:47,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution cfc33a4a114706f42376378024a7423f 2023-02-07 15:55:47,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:47,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for cfc33a4a114706f42376378024a7423f 2023-02-07 15:55:47,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for cfc33a4a114706f42376378024a7423f 2023-02-07 15:55:47,253 INFO [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cfc33a4a114706f42376378024a7423f 2023-02-07 15:55:47,254 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1 2023-02-07 15:55:47,254 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1 2023-02-07 15:55:47,255 INFO [StoreOpener-cfc33a4a114706f42376378024a7423f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cfc33a4a114706f42376378024a7423f columnFamilyName cf1 2023-02-07 15:55:47,273 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897-bottom 2023-02-07 15:55:47,279 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581-bottom 2023-02-07 15:55:47,285 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357-bottom 2023-02-07 15:55:47,291 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f-bottom 2023-02-07 15:55:47,296 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4-bottom 2023-02-07 15:55:47,300 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1-bottom 2023-02-07 15:55:47,305 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb-bottom 2023-02-07 15:55:47,305 INFO [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(310): Store=cfc33a4a114706f42376378024a7423f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:47,306 INFO [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region cfc33a4a114706f42376378024a7423f 2023-02-07 15:55:47,306 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2 2023-02-07 15:55:47,306 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2 2023-02-07 15:55:47,307 INFO [StoreOpener-cfc33a4a114706f42376378024a7423f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cfc33a4a114706f42376378024a7423f columnFamilyName cf2 2023-02-07 15:55:47,323 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865-bottom 2023-02-07 15:55:47,327 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a-bottom 2023-02-07 15:55:47,332 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a-bottom 2023-02-07 15:55:47,336 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5-bottom 2023-02-07 15:55:47,339 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af-bottom 2023-02-07 15:55:47,343 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a-bottom 2023-02-07 15:55:47,347 DEBUG [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9-bottom 2023-02-07 15:55:47,347 INFO [StoreOpener-cfc33a4a114706f42376378024a7423f-1] regionserver.HStore(310): Store=cfc33a4a114706f42376378024a7423f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:47,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f 2023-02-07 15:55:47,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f 2023-02-07 15:55:47,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:55:47,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for cfc33a4a114706f42376378024a7423f 2023-02-07 15:55:47,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened cfc33a4a114706f42376378024a7423f; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74651814, jitterRate=0.11239871382713318}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:55:47,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for cfc33a4a114706f42376378024a7423f: Running coprocessor pre-open hook at 1675785347251Writing region info on filesystem at 1675785347251Initializing all the Stores at 1675785347252 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785347252Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785347253 (+1 ms)Cleaning up temporary data from old regions at 1675785347349 (+96 ms)Cleaning up detritus from prior splits at 1675785347349Running coprocessor post-open hooks at 1675785347351 (+2 ms)Region opened successfully at 1675785347351 2023-02-07 15:55:47,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f., pid=117, masterSystemTime=1675785347244 2023-02-07 15:55:47,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:47,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:55:47,354 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:55:47,355 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:55:47,355 INFO [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f. 2023-02-07 15:55:47,355 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HStore(1912): cfc33a4a114706f42376378024a7423f/cf1 is initiating minor compaction (all files) 2023-02-07 15:55:47,355 INFO [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f. 2023-02-07 15:55:47,355 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HStore(1912): cfc33a4a114706f42376378024a7423f/cf2 is initiating minor compaction (all files) 2023-02-07 15:55:47,355 INFO [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of cfc33a4a114706f42376378024a7423f/cf1 in testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f. 2023-02-07 15:55:47,355 INFO [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HRegion(2258): Starting compaction of cfc33a4a114706f42376378024a7423f/cf2 in testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f. 2023-02-07 15:55:47,355 INFO [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581-bottom] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/.tmp, totalSize=34.9 K 2023-02-07 15:55:47,355 INFO [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a-bottom] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/.tmp, totalSize=34.9 K 2023-02-07 15:55:47,356 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting 1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785339798 2023-02-07 15:55:47,356 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785339798 2023-02-07 15:55:47,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f. 2023-02-07 15:55:47,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f. 2023-02-07 15:55:47,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace. 2023-02-07 15:55:47,356 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785340027 2023-02-07 15:55:47,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 48fead1a7a8db08778f1e1e72513eace, NAME => 'testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.', STARTKEY => '41', ENDKEY => ''} 2023-02-07 15:55:47,356 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting 084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785340027 2023-02-07 15:55:47,356 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=cfc33a4a114706f42376378024a7423f, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:47,356 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting 5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785340331 2023-02-07 15:55:47,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:55:47,356 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785340331 2023-02-07 15:55:47,356 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675785347356"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785347356"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785347356"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785347356"}]},"ts":"1675785347356"} 2023-02-07 15:55:47,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:47,357 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785340627 2023-02-07 15:55:47,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,357 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting 5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785340627 2023-02-07 15:55:47,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,357 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785340879 2023-02-07 15:55:47,357 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting 8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785340879 2023-02-07 15:55:47,357 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785341152 2023-02-07 15:55:47,358 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting 84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785341152 2023-02-07 15:55:47,358 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting 36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675785341456 2023-02-07 15:55:47,358 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting 4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675785341456 2023-02-07 15:55:47,358 INFO [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,359 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1 2023-02-07 15:55:47,360 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1 2023-02-07 15:55:47,360 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-02-07 15:55:47,360 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure cfc33a4a114706f42376378024a7423f, server=jenkins-hbase12.apache.org,37399,1675785236898 in 265 msec 2023-02-07 15:55:47,360 INFO [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 48fead1a7a8db08778f1e1e72513eace columnFamilyName cf1 2023-02-07 15:55:47,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=cfc33a4a114706f42376378024a7423f, ASSIGN in 426 msec 2023-02-07 15:55:47,367 INFO [RS:2;jenkins-hbase12:37399-longCompactions-0] throttle.PressureAwareThroughputController(145): cfc33a4a114706f42376378024a7423f#cf2#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:55:47,367 INFO [RS:2;jenkins-hbase12:37399-shortCompactions-0] throttle.PressureAwareThroughputController(145): cfc33a4a114706f42376378024a7423f#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:55:47,374 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897-top 2023-02-07 15:55:47,378 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581-top 2023-02-07 15:55:47,380 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/.tmp/cf2/5f88863b9f944662b4812ddf02b80c47 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/5f88863b9f944662b4812ddf02b80c47 2023-02-07 15:55:47,383 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-07 15:55:47,384 INFO [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cfc33a4a114706f42376378024a7423f/cf2 of cfc33a4a114706f42376378024a7423f into 5f88863b9f944662b4812ddf02b80c47(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:55:47,384 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for cfc33a4a114706f42376378024a7423f: 2023-02-07 15:55:47,384 INFO [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f., storeName=cfc33a4a114706f42376378024a7423f/cf2, priority=9, startTime=1675785347352; duration=0sec 2023-02-07 15:55:47,384 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:47,389 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357-top 2023-02-07 15:55:47,392 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f-top 2023-02-07 15:55:47,392 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/.tmp/cf1/4ede49c1e87143b4abe0083a139fce3c as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/4ede49c1e87143b4abe0083a139fce3c 2023-02-07 15:55:47,395 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4-top 2023-02-07 15:55:47,397 INFO [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cfc33a4a114706f42376378024a7423f/cf1 of cfc33a4a114706f42376378024a7423f into 4ede49c1e87143b4abe0083a139fce3c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:55:47,397 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for cfc33a4a114706f42376378024a7423f: 2023-02-07 15:55:47,397 INFO [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f., storeName=cfc33a4a114706f42376378024a7423f/cf1, priority=9, startTime=1675785347352; duration=0sec 2023-02-07 15:55:47,397 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:47,402 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1-top 2023-02-07 15:55:47,406 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb-top 2023-02-07 15:55:47,406 INFO [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(310): Store=48fead1a7a8db08778f1e1e72513eace/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:47,406 INFO [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,407 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2 2023-02-07 15:55:47,407 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2 2023-02-07 15:55:47,407 INFO [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 48fead1a7a8db08778f1e1e72513eace columnFamilyName cf2 2023-02-07 15:55:47,420 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865-top 2023-02-07 15:55:47,425 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a-top 2023-02-07 15:55:47,429 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a-top 2023-02-07 15:55:47,435 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5-top 2023-02-07 15:55:47,438 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af-top 2023-02-07 15:55:47,441 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a-top 2023-02-07 15:55:47,444 DEBUG [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9-top 2023-02-07 15:55:47,445 INFO [StoreOpener-48fead1a7a8db08778f1e1e72513eace-1] regionserver.HStore(310): Store=48fead1a7a8db08778f1e1e72513eace/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:47,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:55:47,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 48fead1a7a8db08778f1e1e72513eace; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59182830, jitterRate=-0.11810711026191711}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:55:47,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 48fead1a7a8db08778f1e1e72513eace: Running coprocessor pre-open hook at 1675785347357Writing region info on filesystem at 1675785347357Initializing all the Stores at 1675785347358 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785347358Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785347358Cleaning up temporary data from old regions at 1675785347446 (+88 ms)Cleaning up detritus from prior splits at 1675785347446Running coprocessor post-open hooks at 1675785347448 (+2 ms)Region opened successfully at 1675785347448 2023-02-07 15:55:47,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace., pid=118, masterSystemTime=1675785347244 2023-02-07 15:55:47,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:47,450 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:55:47,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:55:47,450 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:55:47,450 INFO [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace. 2023-02-07 15:55:47,450 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HStore(1912): 48fead1a7a8db08778f1e1e72513eace/cf1 is initiating minor compaction (all files) 2023-02-07 15:55:47,450 INFO [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace. 2023-02-07 15:55:47,450 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HStore(1912): 48fead1a7a8db08778f1e1e72513eace/cf2 is initiating minor compaction (all files) 2023-02-07 15:55:47,450 INFO [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 48fead1a7a8db08778f1e1e72513eace/cf1 in testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace. 2023-02-07 15:55:47,450 INFO [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 48fead1a7a8db08778f1e1e72513eace/cf2 in testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace. 2023-02-07 15:55:47,451 INFO [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581-top] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/.tmp, totalSize=34.9 K 2023-02-07 15:55:47,451 INFO [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a-top] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/.tmp, totalSize=34.9 K 2023-02-07 15:55:47,451 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting 1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675785339798 2023-02-07 15:55:47,451 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675785339798 2023-02-07 15:55:47,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace. 2023-02-07 15:55:47,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace. 2023-02-07 15:55:47,452 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=48fead1a7a8db08778f1e1e72513eace, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:47,452 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675785347452"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785347452"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785347452"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785347452"}]},"ts":"1675785347452"} 2023-02-07 15:55:47,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-07 15:55:47,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 48fead1a7a8db08778f1e1e72513eace, server=jenkins-hbase12.apache.org,37399,1675785236898 in 355 msec 2023-02-07 15:55:47,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-02-07 15:55:47,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48fead1a7a8db08778f1e1e72513eace, ASSIGN in 520 msec 2023-02-07 15:55:47,455 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 15:55:47,455 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:55:47,455 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 15:55:47,456 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675785340027 2023-02-07 15:55:47,456 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting 084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675785340027 2023-02-07 15:55:47,456 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting 5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675785340331 2023-02-07 15:55:47,456 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675785340331 2023-02-07 15:55:47,456 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675785340627 2023-02-07 15:55:47,456 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting 5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675785340627 2023-02-07 15:55:47,456 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675785340879 2023-02-07 15:55:47,457 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting 8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675785340879 2023-02-07 15:55:47,457 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675785341152 2023-02-07 15:55:47,457 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting 84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675785341152 2023-02-07 15:55:47,457 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] compactions.Compactor(207): Compacting 36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675785341456 2023-02-07 15:55:47,457 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] compactions.Compactor(207): Compacting 4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675785341456 2023-02-07 15:55:47,465 INFO [RS:2;jenkins-hbase12:37399-shortCompactions-0] throttle.PressureAwareThroughputController(145): 48fead1a7a8db08778f1e1e72513eace#cf1#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:55:47,465 INFO [RS:2;jenkins-hbase12:37399-longCompactions-0] throttle.PressureAwareThroughputController(145): 48fead1a7a8db08778f1e1e72513eace#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:55:47,484 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/.tmp/cf1/d1dff31d8edb4356861880d0743bfd94 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/d1dff31d8edb4356861880d0743bfd94 2023-02-07 15:55:47,484 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/.tmp/cf2/1fdc3614710546509c2cf3bfd088dff7 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/1fdc3614710546509c2cf3bfd088dff7 2023-02-07 15:55:47,488 INFO [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 48fead1a7a8db08778f1e1e72513eace/cf2 of 48fead1a7a8db08778f1e1e72513eace into 1fdc3614710546509c2cf3bfd088dff7(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:55:47,488 INFO [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 48fead1a7a8db08778f1e1e72513eace/cf1 of 48fead1a7a8db08778f1e1e72513eace into d1dff31d8edb4356861880d0743bfd94(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:55:47,488 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 48fead1a7a8db08778f1e1e72513eace: 2023-02-07 15:55:47,488 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 48fead1a7a8db08778f1e1e72513eace: 2023-02-07 15:55:47,488 INFO [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace., storeName=48fead1a7a8db08778f1e1e72513eace/cf2, priority=9, startTime=1675785347450; duration=0sec 2023-02-07 15:55:47,488 INFO [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace., storeName=48fead1a7a8db08778f1e1e72513eace/cf1, priority=9, startTime=1675785347449; duration=0sec 2023-02-07 15:55:47,488 DEBUG [RS:2;jenkins-hbase12:37399-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:47,488 DEBUG [RS:2;jenkins-hbase12:37399-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:47,628 ERROR [BP-1532654085-136.243.104.168-1675785230655 heartbeating to localhost.localdomain/127.0.0.1:43555] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1532654085-136.243.104.168-1675785230655 (Datanode Uuid 8f3909c3-b4fb-4933-8c65-751b48eaa405) service to localhost.localdomain/127.0.0.1:43555 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 15:55:47,684 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,684 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 15:55:47,684 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:55:47,685 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:55:47,685 ERROR [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 15:55:47,685 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 15:55:47,685 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 15:55:47,685 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 15:55:47,685 INFO [Listener at localhost.localdomain/41965] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 15:55:47,686 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 15:55:47,687 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-07 15:55:47,687 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 15:55:47,687 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 15:55:47,687 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 21] 2023-02-07 15:55:47,687 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-07 15:55:47,687 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 23] 2023-02-07 15:55:47,687 DEBUG [Listener at localhost.localdomain/41965] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-07 15:55:47,687 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 15:55:47,688 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 15:55:47,688 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 15:55:47,688 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 15:55:47,688 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:47,688 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1e613dcf57a1f9c06070a32173e39028, daughterA=065cd7fcb400a33afc751c65a91aa196, daughterB=7cc858c64b5de4b5ff9aa3f61d757dee 2023-02-07 15:55:47,688 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:47,688 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 15:55:47,688 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:47,689 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=d7d21b347a5059d92fcfbcd4fc307543, daughterA=0c9f79cf0473a30691a3c2d31c830aa7, daughterB=fadaaa28c739d279eb5c9fd9b2befe27 2023-02-07 15:55:47,689 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:47,689 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 15:55:47,689 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:47,689 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=d49006b752d6e5dc24676c9215eaa7dc, daughterA=b992767259acdadc811d9727ba201a05, daughterB=c1f9597ce00dcf3521f395fe441cc65c 2023-02-07 15:55:47,689 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=d49006b752d6e5dc24676c9215eaa7dc, ASSIGN 2023-02-07 15:55:47,689 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:47,689 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 15:55:47,690 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:47,690 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3c6c8eb4e621e1f103ed5aa628afa6b2, daughterA=1bdbb1dd25cc1c811bd033b566adb3f8, daughterB=5866b03ffd45bb7a8602b40ca7d13ab0 2023-02-07 15:55:47,690 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:47,690 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 15:55:47,690 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:47,690 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=33148689d0ba6f95ebbfc74509e65344, daughterA=8484d4fdd12a9c6089324f81f4671ae7, daughterB=1b6585777b9c8a7031889d672dbf4d81 2023-02-07 15:55:47,690 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:47,690 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 15:55:47,690 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:47,691 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:47,691 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRollbackForSplitTableRegionWithReplica, region=adc6382420bc698237a8a83071387a40 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=b0c44c23951518cf7a7c968043412542, daughterA=f9ad60a17c9a9b6f91943158faf8d1fa, daughterB=d48054f26cfb14aa7283cd8e4ae2bbb4 2023-02-07 15:55:47,691 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:47,691 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 15:55:47,691 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:47,691 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:47,691 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 15:55:47,691 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:47,692 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,692 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN 2023-02-07 15:55:47,692 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:47,692 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=cfc33a4a114706f42376378024a7423f, ASSIGN 2023-02-07 15:55:47,692 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48fead1a7a8db08778f1e1e72513eace, ASSIGN 2023-02-07 15:55:47,692 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure cfc33a4a114706f42376378024a7423f, server=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:47,693 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 48fead1a7a8db08778f1e1e72513eace, server=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:47,693 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace held the lock before restarting, call acquireLock to restore it. 2023-02-07 15:55:47,693 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,693 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,693 INFO [Listener at localhost.localdomain/41965] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,693 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:47,693 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 0f24d22640b51f10471c5d20c64f2a00, server=jenkins-hbase12.apache.org,37399,1675785236898 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:47,693 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=cfc33a4a114706f42376378024a7423f, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:47,693 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48fead1a7a8db08778f1e1e72513eace, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:47,693 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure cfc33a4a114706f42376378024a7423f, server=jenkins-hbase12.apache.org,37399,1675785236898 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:47,693 DEBUG [Listener at localhost.localdomain/41965] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 48fead1a7a8db08778f1e1e72513eace, server=jenkins-hbase12.apache.org,37399,1675785236898 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 15:55:47,693 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-07 15:55:47,694 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 15:55:47,695 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,37399,1675785236898, table=hbase:meta, region=1588230740 2023-02-07 15:55:47,695 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675785377695 2023-02-07 15:55:47,696 DEBUG [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 15:55:47,696 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:47,699 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=2b3e2412ef7ea40bb26319d571a3f2e6, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,38629,1675785236797, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797, openSeqNum=2 2023-02-07 15:55:47,699 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=0f24d22640b51f10471c5d20c64f2a00, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=2 2023-02-07 15:55:47,699 INFO [Listener at localhost.localdomain/41965] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,37399,1675785236898, table=testRecoveryAndDoubleExecution, region=0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:47,699 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=cfc33a4a114706f42376378024a7423f, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=86 2023-02-07 15:55:47,700 DEBUG [Listener at localhost.localdomain/41965] assignment.RegionStateStore(147): Load hbase:meta entry region=48fead1a7a8db08778f1e1e72513eace, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37399,1675785236898, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898, openSeqNum=86 2023-02-07 15:55:47,700 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 15:55:47,700 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675785407700 2023-02-07 15:55:47,700 INFO [Listener at localhost.localdomain/41965] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675785467700 2023-02-07 15:55:47,700 INFO [Listener at localhost.localdomain/41965] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 15:55:47,700 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 15:55:47,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=0f24d22640b51f10471c5d20c64f2a00, daughterA=cfc33a4a114706f42376378024a7423f, daughterB=48fead1a7a8db08778f1e1e72513eace in 5.9860 sec 2023-02-07 15:55:47,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37399] ipc.CallRunner(144): callId: 862 service: ClientService methodName: Scan size: 154 connection: 136.243.104.168:39832 deadline: 1675785407951, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00. is not online on jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:48,070 DEBUG [Listener at localhost.localdomain/41965] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 15:55:48,079 INFO [Listener at localhost.localdomain/41965] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-07 15:55:48,082 DEBUG [Listener at localhost.localdomain/41965] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 15:55:48,091 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:55:48,092 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:48,092 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:48,093 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:48,093 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:48,093 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-07 15:55:48,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testRecoveryAndDoubleExecution 2023-02-07 15:55:48,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:48,095 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:48,095 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:48,096 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785348096"}]},"ts":"1675785348096"} 2023-02-07 15:55:48,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-07 15:55:48,097 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-07 15:55:48,151 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-07 15:55:48,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=cfc33a4a114706f42376378024a7423f, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48fead1a7a8db08778f1e1e72513eace, UNASSIGN}] 2023-02-07 15:55:48,154 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48fead1a7a8db08778f1e1e72513eace, UNASSIGN 2023-02-07 15:55:48,155 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=48fead1a7a8db08778f1e1e72513eace, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:48,155 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675785348155"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785348155"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785348155"}]},"ts":"1675785348155"} 2023-02-07 15:55:48,158 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 48fead1a7a8db08778f1e1e72513eace, server=jenkins-hbase12.apache.org,37399,1675785236898}] 2023-02-07 15:55:48,162 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=cfc33a4a114706f42376378024a7423f, UNASSIGN 2023-02-07 15:55:48,162 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=cfc33a4a114706f42376378024a7423f, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:48,163 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675785348162"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785348162"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785348162"}]},"ts":"1675785348162"} 2023-02-07 15:55:48,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure cfc33a4a114706f42376378024a7423f, server=jenkins-hbase12.apache.org,37399,1675785236898}] 2023-02-07 15:55:48,196 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:48,196 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:48,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-07 15:55:48,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:48,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 48fead1a7a8db08778f1e1e72513eace, disabling compactions & flushes 2023-02-07 15:55:48,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace. 2023-02-07 15:55:48,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace. 2023-02-07 15:55:48,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace. after waiting 0 ms 2023-02-07 15:55:48,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace. 2023-02-07 15:55:48,312 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581-top] to archive 2023-02-07 15:55:48,312 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:55:48,315 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,316 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,318 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,319 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,320 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,322 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,323 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,324 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a-top] to archive 2023-02-07 15:55:48,325 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:55:48,327 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,329 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,330 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,331 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,332 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,333 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,333 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 15:55:48,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:55:48,337 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace. 2023-02-07 15:55:48,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 48fead1a7a8db08778f1e1e72513eace: Waiting for close lock at 1675785348311Running coprocessor pre-close hooks at 1675785348311Disabling compacts and flushes for region at 1675785348311Disabling writes for close at 1675785348311Writing region close event to WAL at 1675785348334 (+23 ms)Running coprocessor post-close hooks at 1675785348337 (+3 ms)Closed at 1675785348337 2023-02-07 15:55:48,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:48,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close cfc33a4a114706f42376378024a7423f 2023-02-07 15:55:48,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing cfc33a4a114706f42376378024a7423f, disabling compactions & flushes 2023-02-07 15:55:48,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f. 2023-02-07 15:55:48,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f. 2023-02-07 15:55:48,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f. after waiting 0 ms 2023-02-07 15:55:48,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f. 2023-02-07 15:55:48,339 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=48fead1a7a8db08778f1e1e72513eace, regionState=CLOSED 2023-02-07 15:55:48,339 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675785348339"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785348339"}]},"ts":"1675785348339"} 2023-02-07 15:55:48,341 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581-bottom] to archive 2023-02-07 15:55:48,341 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:55:48,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-07 15:55:48,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 48fead1a7a8db08778f1e1e72513eace, server=jenkins-hbase12.apache.org,37399,1675785236898 in 183 msec 2023-02-07 15:55:48,343 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48fead1a7a8db08778f1e1e72513eace, UNASSIGN in 189 msec 2023-02-07 15:55:48,345 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,346 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,347 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,348 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,350 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,351 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,352 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a-bottom] to archive 2023-02-07 15:55:48,353 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:55:48,355 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,356 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,357 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,358 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,359 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,360 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,361 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 15:55:48,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:55:48,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f. 2023-02-07 15:55:48,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for cfc33a4a114706f42376378024a7423f: Waiting for close lock at 1675785348339Running coprocessor pre-close hooks at 1675785348339Disabling compacts and flushes for region at 1675785348339Disabling writes for close at 1675785348339Writing region close event to WAL at 1675785348362 (+23 ms)Running coprocessor post-close hooks at 1675785348364 (+2 ms)Closed at 1675785348364 2023-02-07 15:55:48,366 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed cfc33a4a114706f42376378024a7423f 2023-02-07 15:55:48,366 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=cfc33a4a114706f42376378024a7423f, regionState=CLOSED 2023-02-07 15:55:48,367 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675785348366"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785348366"}]},"ts":"1675785348366"} 2023-02-07 15:55:48,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-07 15:55:48,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure cfc33a4a114706f42376378024a7423f, server=jenkins-hbase12.apache.org,37399,1675785236898 in 204 msec 2023-02-07 15:55:48,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-07 15:55:48,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=cfc33a4a114706f42376378024a7423f, UNASSIGN in 216 msec 2023-02-07 15:55:48,371 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785348371"}]},"ts":"1675785348371"} 2023-02-07 15:55:48,372 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-07 15:55:48,384 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-07 15:55:48,386 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 292 msec 2023-02-07 15:55:48,398 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:48,398 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:48,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-07 15:55:48,400 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-07 15:55:48,400 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:48,400 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:48,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testRecoveryAndDoubleExecution 2023-02-07 15:55:48,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:48,405 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:48,405 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:48,405 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:48,406 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:48,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-07 15:55:48,414 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,414 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f 2023-02-07 15:55:48,414 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:48,417 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/recovered.edits] 2023-02-07 15:55:48,418 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/recovered.edits] 2023-02-07 15:55:48,418 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/recovered.edits] 2023-02-07 15:55:48,422 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/4ede49c1e87143b4abe0083a139fce3c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf1/4ede49c1e87143b4abe0083a139fce3c 2023-02-07 15:55:48,423 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/.regioninfo to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/.regioninfo 2023-02-07 15:55:48,423 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/1e962a23ac2d4a6a918e473a97bd7897 2023-02-07 15:55:48,425 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/36a22297520746069727bf9f12d7a581 2023-02-07 15:55:48,425 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/5f88863b9f944662b4812ddf02b80c47 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cf2/5f88863b9f944662b4812ddf02b80c47 2023-02-07 15:55:48,426 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/5e46ea137223454d916714ad7a522357 2023-02-07 15:55:48,426 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,427 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/ae9d35e519d74f0894c2de4caa51300f 2023-02-07 15:55:48,427 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,428 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/.regioninfo to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/.regioninfo 2023-02-07 15:55:48,428 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/bae7176e038244d5a366c0b4f9a8d2f4 2023-02-07 15:55:48,428 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,430 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/caf908aec3d24f9497871245c2a466e1 2023-02-07 15:55:48,430 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,432 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf1/d15ceae042244d1587dfd9c4fb2743eb 2023-02-07 15:55:48,432 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,433 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/1e962a23ac2d4a6a918e473a97bd7897.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,434 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,434 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/36a22297520746069727bf9f12d7a581.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,435 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,435 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/084b046260fc45a28fc2617ff0951865 2023-02-07 15:55:48,435 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/5e46ea137223454d916714ad7a522357.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,436 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/4787d0f13eee4eac82743f6c164f999a 2023-02-07 15:55:48,437 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/ae9d35e519d74f0894c2de4caa51300f.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,437 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,437 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/5014a511e4804a50b6a77dff3215a44a 2023-02-07 15:55:48,438 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/bae7176e038244d5a366c0b4f9a8d2f4.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,438 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,438 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/84abd4ec29984e5790af488048f874c5 2023-02-07 15:55:48,439 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/caf908aec3d24f9497871245c2a466e1.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,441 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,441 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/8e2147f6cb7a4cc7a55e76ea182e93af 2023-02-07 15:55:48,442 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf1/d15ceae042244d1587dfd9c4fb2743eb.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,443 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,443 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/d52762de014247ab93fb1632b213da2a 2023-02-07 15:55:48,444 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,444 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/cf2/f0cc35d41bc74940837ec44ef16760a9 2023-02-07 15:55:48,445 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/084b046260fc45a28fc2617ff0951865.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,445 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,446 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/4787d0f13eee4eac82743f6c164f999a.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,446 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/recovered.edits/85.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00/recovered.edits/85.seqid 2023-02-07 15:55:48,447 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/48fead1a7a8db08778f1e1e72513eace/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,447 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,447 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/5014a511e4804a50b6a77dff3215a44a.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,448 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/84abd4ec29984e5790af488048f874c5.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,448 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/d1dff31d8edb4356861880d0743bfd94 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf1/d1dff31d8edb4356861880d0743bfd94 2023-02-07 15:55:48,449 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/8e2147f6cb7a4cc7a55e76ea182e93af.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,450 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/d52762de014247ab93fb1632b213da2a.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,450 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/1fdc3614710546509c2cf3bfd088dff7 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/cf2/1fdc3614710546509c2cf3bfd088dff7 2023-02-07 15:55:48,451 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/cfc33a4a114706f42376378024a7423f/cf2/f0cc35d41bc74940837ec44ef16760a9.0f24d22640b51f10471c5d20c64f2a00 2023-02-07 15:55:48,452 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/recovered.edits/90.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace/recovered.edits/90.seqid 2023-02-07 15:55:48,453 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/recovered.edits/90.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f/recovered.edits/90.seqid 2023-02-07 15:55:48,453 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/48fead1a7a8db08778f1e1e72513eace 2023-02-07 15:55:48,453 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testRecoveryAndDoubleExecution/cfc33a4a114706f42376378024a7423f 2023-02-07 15:55:48,453 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-07 15:55:48,455 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:48,456 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-07 15:55:48,458 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-07 15:55:48,459 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:48,459 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-07 15:55:48,459 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785348459"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:48,459 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785348459"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:48,459 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785348459"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:48,460 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 15:55:48,460 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0f24d22640b51f10471c5d20c64f2a00, NAME => 'testRecoveryAndDoubleExecution,,1675785339291.0f24d22640b51f10471c5d20c64f2a00.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => cfc33a4a114706f42376378024a7423f, NAME => 'testRecoveryAndDoubleExecution,,1675785341713.cfc33a4a114706f42376378024a7423f.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 48fead1a7a8db08778f1e1e72513eace, NAME => 'testRecoveryAndDoubleExecution,41,1675785341713.48fead1a7a8db08778f1e1e72513eace.', STARTKEY => '41', ENDKEY => ''}] 2023-02-07 15:55:48,460 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-07 15:55:48,461 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675785348460"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:48,462 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-07 15:55:48,507 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:48,508 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:48,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-07 15:55:48,511 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 15:55:48,513 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 109 msec 2023-02-07 15:55:48,709 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:48,709 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:48,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-07 15:55:48,711 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-07 15:55:48,732 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=374 (was 368) - Thread LEAK? -, OpenFileDescriptor=847 (was 742) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=188 (was 213), ProcessCount=174 (was 174), AvailableMemoryMB=3014 (was 3078) 2023-02-07 15:55:48,745 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=374, OpenFileDescriptor=847, MaxFileDescriptor=60000, SystemLoadAverage=188, ProcessCount=174, AvailableMemoryMB=3014 2023-02-07 15:55:48,745 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:55:48,746 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:48,746 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:48,752 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 15:55:48,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 15:55:48,754 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 15:55:48,755 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 15:55:48,755 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 15:55:48,757 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:48,757 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e empty. 2023-02-07 15:55:48,758 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:48,758 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-07 15:55:48,770 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-07 15:55:48,772 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 423dda2174efcce0795a0499add9f89e, NAME => 'testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp 2023-02-07 15:55:48,778 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:48,778 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 423dda2174efcce0795a0499add9f89e, disabling compactions & flushes 2023-02-07 15:55:48,778 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:48,778 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:48,779 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. after waiting 0 ms 2023-02-07 15:55:48,779 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:48,779 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:48,779 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 423dda2174efcce0795a0499add9f89e: Waiting for close lock at 1675785348778Disabling compacts and flushes for region at 1675785348778Disabling writes for close at 1675785348779 (+1 ms)Writing region close event to WAL at 1675785348779Closed at 1675785348779 2023-02-07 15:55:48,780 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 15:55:48,781 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785348781"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785348781"}]},"ts":"1675785348781"} 2023-02-07 15:55:48,782 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 15:55:48,782 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 15:55:48,783 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785348782"}]},"ts":"1675785348782"} 2023-02-07 15:55:48,783 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-07 15:55:48,805 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:55:48,806 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:55:48,806 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:55:48,806 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:55:48,806 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 15:55:48,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=423dda2174efcce0795a0499add9f89e, ASSIGN}] 2023-02-07 15:55:48,808 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=423dda2174efcce0795a0499add9f89e, ASSIGN 2023-02-07 15:55:48,809 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=423dda2174efcce0795a0499add9f89e, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:55:48,959 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:55:48,961 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=423dda2174efcce0795a0499add9f89e, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:48,961 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785348961"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785348961"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785348961"}]},"ts":"1675785348961"} 2023-02-07 15:55:48,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 423dda2174efcce0795a0499add9f89e, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:49,072 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 15:55:49,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:49,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 423dda2174efcce0795a0499add9f89e, NAME => 'testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:55:49,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:55:49,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:49,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:49,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:49,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:49,124 INFO [StoreOpener-423dda2174efcce0795a0499add9f89e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:49,125 DEBUG [StoreOpener-423dda2174efcce0795a0499add9f89e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1 2023-02-07 15:55:49,126 DEBUG [StoreOpener-423dda2174efcce0795a0499add9f89e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1 2023-02-07 15:55:49,126 INFO [StoreOpener-423dda2174efcce0795a0499add9f89e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 423dda2174efcce0795a0499add9f89e columnFamilyName cf1 2023-02-07 15:55:49,126 INFO [StoreOpener-423dda2174efcce0795a0499add9f89e-1] regionserver.HStore(310): Store=423dda2174efcce0795a0499add9f89e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:49,126 INFO [StoreOpener-423dda2174efcce0795a0499add9f89e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:49,128 DEBUG [StoreOpener-423dda2174efcce0795a0499add9f89e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2 2023-02-07 15:55:49,128 DEBUG [StoreOpener-423dda2174efcce0795a0499add9f89e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2 2023-02-07 15:55:49,128 INFO [StoreOpener-423dda2174efcce0795a0499add9f89e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 423dda2174efcce0795a0499add9f89e columnFamilyName cf2 2023-02-07 15:55:49,128 INFO [StoreOpener-423dda2174efcce0795a0499add9f89e-1] regionserver.HStore(310): Store=423dda2174efcce0795a0499add9f89e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:49,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:49,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:49,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:55:49,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:49,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:55:49,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 423dda2174efcce0795a0499add9f89e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68799044, jitterRate=0.02518564462661743}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:55:49,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 423dda2174efcce0795a0499add9f89e: Running coprocessor pre-open hook at 1675785349123Writing region info on filesystem at 1675785349123Initializing all the Stores at 1675785349124 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785349124Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785349124Cleaning up temporary data from old regions at 1675785349130 (+6 ms)Cleaning up detritus from prior splits at 1675785349130Running coprocessor post-open hooks at 1675785349133 (+3 ms)Region opened successfully at 1675785349133 2023-02-07 15:55:49,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e., pid=127, masterSystemTime=1675785349118 2023-02-07 15:55:49,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:49,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:49,135 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=423dda2174efcce0795a0499add9f89e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,135 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785349135"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785349135"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785349135"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785349135"}]},"ts":"1675785349135"} 2023-02-07 15:55:49,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-07 15:55:49,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 423dda2174efcce0795a0499add9f89e, server=jenkins-hbase12.apache.org,38629,1675785236797 in 173 msec 2023-02-07 15:55:49,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-07 15:55:49,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=423dda2174efcce0795a0499add9f89e, ASSIGN in 331 msec 2023-02-07 15:55:49,147 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 15:55:49,147 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785349147"}]},"ts":"1675785349147"} 2023-02-07 15:55:49,148 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-07 15:55:49,153 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 15:55:49,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 400 msec 2023-02-07 15:55:49,263 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:49,264 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:49,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:49,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,266 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:49,266 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:49,266 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-07 15:55:49,266 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-07 15:55:49,267 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,267 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,279 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:49,279 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:49,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:49,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:49,279 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:49,279 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:49,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:49,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:49,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:49,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,279 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,279 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:49,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:49,281 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,281 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:49,281 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:49,282 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:49,282 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. started... 2023-02-07 15:55:49,283 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 423dda2174efcce0795a0499add9f89e 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 15:55:49,289 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,293 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/4b18c9171b1440ab902e8888e7d1cfc9 2023-02-07 15:55:49,300 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,300 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,300 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:49,300 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,300 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:49,708 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/5450b80b938d43f2b4b0d1ed2a1e176d 2023-02-07 15:55:49,718 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/4b18c9171b1440ab902e8888e7d1cfc9 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/4b18c9171b1440ab902e8888e7d1cfc9 2023-02-07 15:55:49,722 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/4b18c9171b1440ab902e8888e7d1cfc9, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:55:49,722 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/5450b80b938d43f2b4b0d1ed2a1e176d as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/5450b80b938d43f2b4b0d1ed2a1e176d 2023-02-07 15:55:49,725 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/5450b80b938d43f2b4b0d1ed2a1e176d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:55:49,726 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 423dda2174efcce0795a0499add9f89e in 443ms, sequenceid=6, compaction requested=false 2023-02-07 15:55:49,726 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:49,726 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 423dda2174efcce0795a0499add9f89e: 2023-02-07 15:55:49,726 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:49,726 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:49,727 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:49,727 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,727 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:49,727 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:49,824 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,824 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:49,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:49,825 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,825 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:49,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:49,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:49,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:49,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:49,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:49,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-07 15:55:49,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69e43b41[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:49,827 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-07 15:55:49,827 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,902 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,902 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,902 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,902 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,902 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:49,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,902 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:49,903 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:49,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,903 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:49,902 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:49,903 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:49,903 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:49,903 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:49,903 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:49,903 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:49,910 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:49,910 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:49,910 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:49,921 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:49,921 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,921 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:49,921 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:49,921 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:49,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:49,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:49,921 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:49,921 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:49,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:49,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:49,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:49,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:49,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:49,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:49,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:49,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:49,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:55:49,931 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-07 15:55:49,931 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:49,931 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:49,931 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,931 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:49,974 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,974 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:49,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:49,974 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,975 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:49,974 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:49,974 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:49,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:49,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:49,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,976 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,976 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,977 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:49,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:49,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:49,978 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:49,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:49,982 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:49,982 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:49,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:49,983 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:50,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:50,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:50,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:50,024 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:50,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:50,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:50,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:50,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:50,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:50,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:50,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:50,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:50,026 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:50,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:50,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:50,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:50,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:50,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,027 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,026 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:50,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:50,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:50,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:50,027 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:50,027 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,127 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:50,127 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:50,128 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:50,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:50,149 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:50,149 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:50,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:50,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:50,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,151 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:50,151 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:50,151 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-07 15:55:50,151 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-07 15:55:50,152 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,152 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,173 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:50,173 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:50,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:50,173 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:50,173 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:50,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:50,174 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,174 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:50,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:50,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:50,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,176 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,175 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,176 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:50,176 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:50,176 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:50,176 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,176 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:50,176 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:50,176 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:50,176 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:50,176 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:50,176 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:50,176 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:50,176 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,176 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:50,176 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:50,176 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:50,176 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:50,177 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:50,177 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:50,177 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:50,177 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:50,177 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. started... 2023-02-07 15:55:50,177 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 423dda2174efcce0795a0499add9f89e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:50,184 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,186 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/542298b43aad4e60948fa03f04b20f99 2023-02-07 15:55:50,194 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,194 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,194 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:50,195 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,195 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:50,197 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/d2e6b41e320f459c86bf67371ba0228e 2023-02-07 15:55:50,201 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/542298b43aad4e60948fa03f04b20f99 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/542298b43aad4e60948fa03f04b20f99 2023-02-07 15:55:50,206 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/542298b43aad4e60948fa03f04b20f99, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:55:50,207 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/d2e6b41e320f459c86bf67371ba0228e as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/d2e6b41e320f459c86bf67371ba0228e 2023-02-07 15:55:50,213 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/d2e6b41e320f459c86bf67371ba0228e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:55:50,214 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 423dda2174efcce0795a0499add9f89e in 37ms, sequenceid=19, compaction requested=false 2023-02-07 15:55:50,214 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 423dda2174efcce0795a0499add9f89e: 2023-02-07 15:55:50,214 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:50,214 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:50,214 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:50,214 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,214 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:50,214 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:50,226 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,226 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:50,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:50,227 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,227 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:50,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:50,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:50,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:50,231 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-07 15:55:50,231 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@221b63d5[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:50,231 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-07 15:55:50,231 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,236 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,237 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,236 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,237 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,237 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:50,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,237 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:50,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,237 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:50,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,237 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:50,237 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:50,237 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:50,237 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:50,237 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:50,238 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:50,237 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:50,247 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:50,247 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,247 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:50,247 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:50,247 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:50,247 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:50,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:50,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:50,247 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:50,247 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:50,247 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:50,247 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:50,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:50,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:50,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:50,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:50,250 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:55:50,250 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-07 15:55:50,250 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:50,250 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:50,250 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,250 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:50,268 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,268 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,268 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:50,268 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,268 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:50,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:50,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,268 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:50,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,268 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:50,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,269 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:50,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:50,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:50,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,269 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:50,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:50,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:50,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:50,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,270 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:50,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,272 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:50,272 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,272 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,289 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:50,289 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:50,289 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:50,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:50,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:50,289 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:50,289 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:50,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:50,289 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:50,289 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:50,289 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:50,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:50,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:50,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:50,290 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:50,290 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:50,291 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:50,290 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:50,291 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:50,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:50,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:50,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:50,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:50,291 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,291 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,291 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,291 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,291 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,292 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,292 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,391 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:50,391 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:50,391 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:50,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:50,410 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:50,410 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:50,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:50,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:50,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,412 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:50,412 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:50,412 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-07 15:55:50,412 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-07 15:55:50,412 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,412 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,424 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,424 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:50,424 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:50,424 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:50,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:50,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:50,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:50,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:50,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:50,425 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:50,425 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:50,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,426 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:50,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:50,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,426 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,426 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:50,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,426 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,426 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:50,426 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:50,426 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,427 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:50,427 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:50,427 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:50,427 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:50,427 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. started... 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:50,427 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:50,427 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 423dda2174efcce0795a0499add9f89e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:50,436 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,439 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/bc26ea27b49b4b77a873fd630b8525ef 2023-02-07 15:55:50,515 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,515 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,515 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:50,516 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,516 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:50,852 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/f5eb0bf8ca8f4f0797ee3456341f8d0d 2023-02-07 15:55:50,860 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/bc26ea27b49b4b77a873fd630b8525ef as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bc26ea27b49b4b77a873fd630b8525ef 2023-02-07 15:55:50,868 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bc26ea27b49b4b77a873fd630b8525ef, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:55:50,869 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/f5eb0bf8ca8f4f0797ee3456341f8d0d as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/f5eb0bf8ca8f4f0797ee3456341f8d0d 2023-02-07 15:55:50,873 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/f5eb0bf8ca8f4f0797ee3456341f8d0d, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:55:50,873 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 423dda2174efcce0795a0499add9f89e in 446ms, sequenceid=32, compaction requested=true 2023-02-07 15:55:50,874 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 423dda2174efcce0795a0499add9f89e: 2023-02-07 15:55:50,874 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:50,874 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:50,874 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:50,874 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,874 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:50,874 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:50,931 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,931 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:50,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:50,932 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,932 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:50,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:50,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:50,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:50,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-07 15:55:50,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d52adbb[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:50,940 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-07 15:55:50,940 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,953 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,953 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,953 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,954 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,954 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:50,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,954 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:50,954 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:50,954 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:50,953 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:50,955 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:50,955 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:50,954 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:50,955 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:50,955 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:50,963 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:50,963 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:50,963 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:50,974 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:50,974 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:50,974 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:50,974 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,974 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:50,975 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:50,975 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:50,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:50,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:50,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:50,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:50,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:50,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:50,986 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:55:50,986 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-07 15:55:50,986 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:50,986 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:50,986 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,986 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:50,994 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,995 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,995 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:50,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,995 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:50,995 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,995 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,995 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:50,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,996 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:50,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:50,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:50,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:50,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:50,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:50,997 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:50,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:50,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:50,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,998 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:50,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:50,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:50,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:50,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:51,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,002 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,002 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,003 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:51,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,015 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,015 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,015 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,015 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,015 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,016 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:51,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:51,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,016 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,016 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,015 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,016 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:51,016 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:51,016 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,016 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:51,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,016 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,016 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,017 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,017 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,017 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,017 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,017 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,116 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:51,117 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:51,117 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:51,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:51,127 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:51,127 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:51,128 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:51,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:51,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,129 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:51,129 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:51,129 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-07 15:55:51,129 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-07 15:55:51,129 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,129 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,142 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,142 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,142 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,142 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,142 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,142 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,143 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:51,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:51,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:51,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:51,144 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:51,144 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:51,144 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. started... 2023-02-07 15:55:51,145 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 423dda2174efcce0795a0499add9f89e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:51,152 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,153 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/d0e0313700634e2b84a939e5a143b14f 2023-02-07 15:55:51,163 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,163 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,163 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:51,163 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,163 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:51,164 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/99272da2a7ed4b0981b7c0aa89a2c020 2023-02-07 15:55:51,168 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/d0e0313700634e2b84a939e5a143b14f as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/d0e0313700634e2b84a939e5a143b14f 2023-02-07 15:55:51,172 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/d0e0313700634e2b84a939e5a143b14f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:55:51,173 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/99272da2a7ed4b0981b7c0aa89a2c020 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/99272da2a7ed4b0981b7c0aa89a2c020 2023-02-07 15:55:51,177 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/99272da2a7ed4b0981b7c0aa89a2c020, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:55:51,178 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 423dda2174efcce0795a0499add9f89e in 33ms, sequenceid=45, compaction requested=true 2023-02-07 15:55:51,178 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 423dda2174efcce0795a0499add9f89e: 2023-02-07 15:55:51,178 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:51,178 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:51,178 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:51,178 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,178 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:51,178 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:51,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,184 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:51,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:51,184 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,184 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:51,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:51,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:51,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:51,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-07 15:55:51,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20195aad[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:51,188 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-07 15:55:51,188 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,194 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,194 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,194 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,195 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,195 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:51,195 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:51,195 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:51,195 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:51,195 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:51,195 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:51,195 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:51,195 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:51,196 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:51,195 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:51,205 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:51,205 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,205 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:51,205 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:51,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:51,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:51,215 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:51,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:51,216 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:51,216 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:51,216 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:51,216 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:51,216 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:51,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:51,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:51,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:51,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:55:51,226 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-07 15:55:51,227 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:51,227 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:51,227 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,227 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:51,237 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,237 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,237 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,237 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,237 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,237 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,237 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,238 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:51,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,239 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:51,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,241 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:51,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:51,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,245 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,246 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,246 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:51,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,299 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,299 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,299 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,299 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,299 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,299 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:51,299 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,299 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,299 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,300 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,299 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter because node does not exist (not an error) 2023-02-07 15:55:51,299 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:51,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:51,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,300 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,300 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:51,300 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,300 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:51,300 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,301 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,301 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,301 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,301 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,401 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:51,401 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:51,401 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:51,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:51,410 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:51,410 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:51,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:51,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:51,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,413 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:51,413 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:51,413 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-07 15:55:51,413 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-07 15:55:51,413 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,413 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,423 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,423 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,423 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,423 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:51,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,424 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:51,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:51,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:51,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:51,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:51,425 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:51,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,425 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:51,425 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:51,425 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:51,425 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:51,426 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:51,425 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:51,426 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:51,426 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:51,425 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:51,426 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:51,426 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:51,426 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:51,426 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:51,426 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:51,426 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:51,426 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. started... 2023-02-07 15:55:51,426 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 423dda2174efcce0795a0499add9f89e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:51,436 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,439 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/20dc002b6d6540cfb6253280b81f7e48 2023-02-07 15:55:51,447 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,447 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:51,447 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,448 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,448 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:51,454 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/cffcf7e92a1e4fc0905a2fbea1e3faf4 2023-02-07 15:55:51,458 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/20dc002b6d6540cfb6253280b81f7e48 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/20dc002b6d6540cfb6253280b81f7e48 2023-02-07 15:55:51,463 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/20dc002b6d6540cfb6253280b81f7e48, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:55:51,465 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/cffcf7e92a1e4fc0905a2fbea1e3faf4 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/cffcf7e92a1e4fc0905a2fbea1e3faf4 2023-02-07 15:55:51,470 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/cffcf7e92a1e4fc0905a2fbea1e3faf4, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:55:51,470 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 423dda2174efcce0795a0499add9f89e in 44ms, sequenceid=58, compaction requested=true 2023-02-07 15:55:51,471 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 423dda2174efcce0795a0499add9f89e: 2023-02-07 15:55:51,471 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:51,471 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:51,471 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:51,471 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,471 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:51,471 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:51,479 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,479 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:51,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:51,479 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,479 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:51,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:51,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:51,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:51,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-07 15:55:51,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@686f43dc[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:51,483 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-07 15:55:51,483 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,489 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,489 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,489 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,490 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,490 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:51,490 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:51,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,490 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:51,490 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:51,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,490 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:51,491 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:51,491 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:51,491 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:51,491 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:51,491 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:51,500 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:51,500 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,500 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:51,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:51,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:51,500 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:51,510 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:51,510 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:51,510 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:51,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:51,521 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:51,521 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:51,521 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:51,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:51,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:51,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:51,525 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:55:51,525 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-07 15:55:51,525 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:51,525 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:51,525 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,525 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:51,531 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,531 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:51,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:51,531 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,531 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,531 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,531 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,531 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,532 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:51,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,533 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,534 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:51,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:51,536 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,536 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,538 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:51,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:51,549 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:51,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,550 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:51,549 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:51,550 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:51,550 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,550 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:51,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:51,550 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,550 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,550 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,550 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,550 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,650 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:51,650 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:51,650 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:51,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:51,664 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:51,664 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:51,665 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:51,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:51,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,668 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:51,668 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:51,668 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-07 15:55:51,668 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-07 15:55:51,669 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,669 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,679 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,679 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,679 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,679 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:51,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,679 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,679 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:51,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:51,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:51,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,680 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,681 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,681 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:51,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,681 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:51,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:51,681 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:51,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,681 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,682 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,682 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,682 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:51,682 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:51,682 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:51,682 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,682 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:51,682 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 15:55:51,682 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:51,682 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:51,682 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:51,682 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,683 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:51,683 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 15:55:51,683 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:51,683 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:51,683 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 15:55:51,683 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:51,683 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:51,683 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:51,684 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:51,684 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:51,684 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. started... 2023-02-07 15:55:51,684 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 423dda2174efcce0795a0499add9f89e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:51,696 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/bb93ddbdbe4946caba401ad89fb1955c 2023-02-07 15:55:51,709 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/1c1c2a3ee8f849e78f1ad3f402339a70 2023-02-07 15:55:51,714 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/bb93ddbdbe4946caba401ad89fb1955c as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bb93ddbdbe4946caba401ad89fb1955c 2023-02-07 15:55:51,719 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bb93ddbdbe4946caba401ad89fb1955c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:55:51,720 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/1c1c2a3ee8f849e78f1ad3f402339a70 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/1c1c2a3ee8f849e78f1ad3f402339a70 2023-02-07 15:55:51,722 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,725 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/1c1c2a3ee8f849e78f1ad3f402339a70, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:55:51,726 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 423dda2174efcce0795a0499add9f89e in 42ms, sequenceid=71, compaction requested=true 2023-02-07 15:55:51,726 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 423dda2174efcce0795a0499add9f89e: 2023-02-07 15:55:51,726 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:51,726 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:51,726 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:51,726 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,726 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 15:55:51,726 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 15:55:51,814 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,814 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,814 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:51,935 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,935 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,935 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:51,935 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:51,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:51,936 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,936 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:51,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:51,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:51,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:51,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-07 15:55:51,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@111390cf[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:51,940 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-07 15:55:51,940 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,970 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,970 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,970 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,971 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,971 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:51,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,971 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:51,971 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:51,971 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:51,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,971 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:51,971 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 15:55:51,971 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:51,971 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:51,971 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 15:55:51,971 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:51,984 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:51,984 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,984 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:51,984 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:51,984 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:51,984 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:51,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:51,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:51,984 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:51,994 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 15:55:51,994 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:51,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:51,994 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 15:55:51,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:51,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:51,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:51,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:51,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:51,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:51,997 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-07 15:55:51,997 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:51,997 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:51,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-07 15:55:51,997 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-07 15:55:51,997 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:52,005 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,005 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,005 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:52,005 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,005 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,005 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:52,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:52,005 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:52,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:52,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:52,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:52,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:52,005 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:52,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:52,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:52,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:52,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:52,007 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:52,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,007 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:52,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:52,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:52,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:52,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:52,009 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:52,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:52,009 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:52,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,009 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:52,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:52,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:52,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:52,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:52,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:52,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:52,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:52,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:52,023 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:52,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:52,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:52,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:52,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:52,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-07 15:55:52,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:52,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:52,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:52,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:52,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:52,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:52,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:52,023 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:52,023 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:52,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:52,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:52,024 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:52,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:52,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:52,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:52,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:52,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:52,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:52,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,024 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 15:55:52,124 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:52,124 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:52,124 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:52,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:52,138 DEBUG [Listener at localhost.localdomain/41965] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:52,139 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=423dda2174efcce0795a0499add9f89e, daughterA=5adddea27dcf0673e88a9fdaa5d479c6, daughterB=c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:52,140 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=423dda2174efcce0795a0499add9f89e, daughterA=5adddea27dcf0673e88a9fdaa5d479c6, daughterB=c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:52,141 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=423dda2174efcce0795a0499add9f89e, daughterA=5adddea27dcf0673e88a9fdaa5d479c6, daughterB=c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:52,141 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=423dda2174efcce0795a0499add9f89e, daughterA=5adddea27dcf0673e88a9fdaa5d479c6, daughterB=c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:52,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=423dda2174efcce0795a0499add9f89e, UNASSIGN}] 2023-02-07 15:55:52,143 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=423dda2174efcce0795a0499add9f89e, UNASSIGN 2023-02-07 15:55:52,144 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=423dda2174efcce0795a0499add9f89e, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:52,144 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785352144"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785352144"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785352144"}]},"ts":"1675785352144"} 2023-02-07 15:55:52,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 423dda2174efcce0795a0499add9f89e, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:52,297 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 423dda2174efcce0795a0499add9f89e, disabling compactions & flushes 2023-02-07 15:55:52,297 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:52,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:52,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. after waiting 0 ms 2023-02-07 15:55:52,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:52,297 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 423dda2174efcce0795a0499add9f89e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 15:55:52,308 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/0fe9f8e4d148440fb8004a2457a11ef8 2023-02-07 15:55:52,321 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/e3569c15560c4344b8aa7800047c2d55 2023-02-07 15:55:52,325 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf1/0fe9f8e4d148440fb8004a2457a11ef8 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/0fe9f8e4d148440fb8004a2457a11ef8 2023-02-07 15:55:52,328 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/0fe9f8e4d148440fb8004a2457a11ef8, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:55:52,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.tmp/cf2/e3569c15560c4344b8aa7800047c2d55 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/e3569c15560c4344b8aa7800047c2d55 2023-02-07 15:55:52,333 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/e3569c15560c4344b8aa7800047c2d55, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:55:52,334 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 423dda2174efcce0795a0499add9f89e in 37ms, sequenceid=82, compaction requested=true 2023-02-07 15:55:52,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 15:55:52,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:55:52,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. 2023-02-07 15:55:52,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 423dda2174efcce0795a0499add9f89e: Waiting for close lock at 1675785352297Running coprocessor pre-close hooks at 1675785352297Disabling compacts and flushes for region at 1675785352297Disabling writes for close at 1675785352297Obtaining lock to block concurrent updates at 1675785352297Preparing flush snapshotting stores in 423dda2174efcce0795a0499add9f89e at 1675785352297Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675785352298 (+1 ms)Flushing stores of testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. at 1675785352298Flushing 423dda2174efcce0795a0499add9f89e/cf1: creating writer at 1675785352300 (+2 ms)Flushing 423dda2174efcce0795a0499add9f89e/cf1: appending metadata at 1675785352302 (+2 ms)Flushing 423dda2174efcce0795a0499add9f89e/cf1: closing flushed file at 1675785352302Flushing 423dda2174efcce0795a0499add9f89e/cf2: creating writer at 1675785352311 (+9 ms)Flushing 423dda2174efcce0795a0499add9f89e/cf2: appending metadata at 1675785352313 (+2 ms)Flushing 423dda2174efcce0795a0499add9f89e/cf2: closing flushed file at 1675785352313Flushing 423dda2174efcce0795a0499add9f89e/cf1: reopening flushed file at 1675785352325 (+12 ms)Flushing 423dda2174efcce0795a0499add9f89e/cf2: reopening flushed file at 1675785352330 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 423dda2174efcce0795a0499add9f89e in 37ms, sequenceid=82, compaction requested=true at 1675785352334 (+4 ms)Writing region close event to WAL at 1675785352336 (+2 ms)Running coprocessor post-close hooks at 1675785352339 (+3 ms)Closed at 1675785352339 2023-02-07 15:55:52,341 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,341 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=423dda2174efcce0795a0499add9f89e, regionState=CLOSED 2023-02-07 15:55:52,341 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785352341"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785352341"}]},"ts":"1675785352341"} 2023-02-07 15:55:52,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-07 15:55:52,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 423dda2174efcce0795a0499add9f89e, server=jenkins-hbase12.apache.org,38629,1675785236797 in 197 msec 2023-02-07 15:55:52,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-07 15:55:52,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=423dda2174efcce0795a0499add9f89e, UNASSIGN in 200 msec 2023-02-07 15:55:52,354 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=423dda2174efcce0795a0499add9f89e, threads=14 2023-02-07 15:55:52,354 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/1c1c2a3ee8f849e78f1ad3f402339a70 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,354 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/5450b80b938d43f2b4b0d1ed2a1e176d for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,355 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/99272da2a7ed4b0981b7c0aa89a2c020 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,355 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/cffcf7e92a1e4fc0905a2fbea1e3faf4 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,355 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/d2e6b41e320f459c86bf67371ba0228e for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,355 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/e3569c15560c4344b8aa7800047c2d55 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,355 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/f5eb0bf8ca8f4f0797ee3456341f8d0d for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,355 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/0fe9f8e4d148440fb8004a2457a11ef8 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,355 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/20dc002b6d6540cfb6253280b81f7e48 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,355 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/4b18c9171b1440ab902e8888e7d1cfc9 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,355 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/542298b43aad4e60948fa03f04b20f99 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,356 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bb93ddbdbe4946caba401ad89fb1955c for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,356 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bc26ea27b49b4b77a873fd630b8525ef for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,356 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/d0e0313700634e2b84a939e5a143b14f for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,358 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/1c1c2a3ee8f849e78f1ad3f402339a70, top=false 2023-02-07 15:55:52,363 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/e3569c15560c4344b8aa7800047c2d55, top=false 2023-02-07 15:55:52,363 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/4b18c9171b1440ab902e8888e7d1cfc9, top=false 2023-02-07 15:55:52,363 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/20dc002b6d6540cfb6253280b81f7e48, top=false 2023-02-07 15:55:52,370 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/5450b80b938d43f2b4b0d1ed2a1e176d, top=false 2023-02-07 15:55:52,370 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/cffcf7e92a1e4fc0905a2fbea1e3faf4, top=false 2023-02-07 15:55:52,372 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/0fe9f8e4d148440fb8004a2457a11ef8, top=false 2023-02-07 15:55:52,373 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bc26ea27b49b4b77a873fd630b8525ef, top=false 2023-02-07 15:55:52,375 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/d0e0313700634e2b84a939e5a143b14f, top=false 2023-02-07 15:55:52,376 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/d2e6b41e320f459c86bf67371ba0228e, top=false 2023-02-07 15:55:52,376 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bb93ddbdbe4946caba401ad89fb1955c, top=false 2023-02-07 15:55:52,378 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/99272da2a7ed4b0981b7c0aa89a2c020, top=false 2023-02-07 15:55:52,378 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/542298b43aad4e60948fa03f04b20f99, top=false 2023-02-07 15:55:52,379 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/f5eb0bf8ca8f4f0797ee3456341f8d0d, top=false 2023-02-07 15:55:52,392 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-4b18c9171b1440ab902e8888e7d1cfc9 for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,392 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-5450b80b938d43f2b4b0d1ed2a1e176d for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,392 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-20dc002b6d6540cfb6253280b81f7e48 for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,392 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-0fe9f8e4d148440fb8004a2457a11ef8 for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,393 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-e3569c15560c4344b8aa7800047c2d55 for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,396 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/20dc002b6d6540cfb6253280b81f7e48 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,403 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/4b18c9171b1440ab902e8888e7d1cfc9 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,403 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/0fe9f8e4d148440fb8004a2457a11ef8 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,404 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/5450b80b938d43f2b4b0d1ed2a1e176d for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,404 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/e3569c15560c4344b8aa7800047c2d55 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,409 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-1c1c2a3ee8f849e78f1ad3f402339a70 for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,409 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-f5eb0bf8ca8f4f0797ee3456341f8d0d for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,409 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bc26ea27b49b4b77a873fd630b8525ef for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,410 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d0e0313700634e2b84a939e5a143b14f for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,410 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d2e6b41e320f459c86bf67371ba0228e for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,415 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/d2e6b41e320f459c86bf67371ba0228e for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,415 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/1c1c2a3ee8f849e78f1ad3f402339a70 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,416 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bc26ea27b49b4b77a873fd630b8525ef for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,417 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/f5eb0bf8ca8f4f0797ee3456341f8d0d for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,418 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/d0e0313700634e2b84a939e5a143b14f for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,420 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-99272da2a7ed4b0981b7c0aa89a2c020 for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,420 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-cffcf7e92a1e4fc0905a2fbea1e3faf4 for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,421 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-542298b43aad4e60948fa03f04b20f99 for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,421 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/.splits/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bb93ddbdbe4946caba401ad89fb1955c for child: 5adddea27dcf0673e88a9fdaa5d479c6, parent: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,423 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bb93ddbdbe4946caba401ad89fb1955c for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,424 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/542298b43aad4e60948fa03f04b20f99 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,424 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/cffcf7e92a1e4fc0905a2fbea1e3faf4 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,424 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/99272da2a7ed4b0981b7c0aa89a2c020 for region: 423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:52,424 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 423dda2174efcce0795a0499add9f89e Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-07 15:55:52,629 ERROR [BP-1532654085-136.243.104.168-1675785230655 heartbeating to localhost.localdomain/127.0.0.1:43555] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1532654085-136.243.104.168-1675785230655 (Datanode Uuid 8f3909c3-b4fb-4933-8c65-751b48eaa405) service to localhost.localdomain/127.0.0.1:43555 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 15:55:52,859 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 15:55:52,860 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 15:55:52,861 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675785352861"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1675785352861"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1675785352861"}]},"ts":"1675785352861"} 2023-02-07 15:55:52,861 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675785352861"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785352861"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785352861"}]},"ts":"1675785352861"} 2023-02-07 15:55:52,861 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675785352861"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785352861"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785352861"}]},"ts":"1675785352861"} 2023-02-07 15:55:52,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5adddea27dcf0673e88a9fdaa5d479c6, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c8abf20a55c734610d779b0a9ffc4e0d, ASSIGN}] 2023-02-07 15:55:52,865 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c8abf20a55c734610d779b0a9ffc4e0d, ASSIGN 2023-02-07 15:55:52,865 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c8abf20a55c734610d779b0a9ffc4e0d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:55:52,865 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5adddea27dcf0673e88a9fdaa5d479c6, ASSIGN 2023-02-07 15:55:52,866 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5adddea27dcf0673e88a9fdaa5d479c6, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:55:53,015 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 15:55:53,018 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=5adddea27dcf0673e88a9fdaa5d479c6, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:53,018 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675785353017"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785353017"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785353017"}]},"ts":"1675785353017"} 2023-02-07 15:55:53,019 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure 5adddea27dcf0673e88a9fdaa5d479c6, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:53,022 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=c8abf20a55c734610d779b0a9ffc4e0d, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:53,022 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675785353022"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785353022"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785353022"}]},"ts":"1675785353022"} 2023-02-07 15:55:53,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure c8abf20a55c734610d779b0a9ffc4e0d, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:53,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d. 2023-02-07 15:55:53,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c8abf20a55c734610d779b0a9ffc4e0d, NAME => 'testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d.', STARTKEY => '71', ENDKEY => ''} 2023-02-07 15:55:53,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:55:53,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:53,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:53,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:53,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:53,178 INFO [StoreOpener-c8abf20a55c734610d779b0a9ffc4e0d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:53,181 DEBUG [StoreOpener-c8abf20a55c734610d779b0a9ffc4e0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d/cf1 2023-02-07 15:55:53,181 DEBUG [StoreOpener-c8abf20a55c734610d779b0a9ffc4e0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d/cf1 2023-02-07 15:55:53,182 INFO [StoreOpener-c8abf20a55c734610d779b0a9ffc4e0d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c8abf20a55c734610d779b0a9ffc4e0d columnFamilyName cf1 2023-02-07 15:55:53,182 INFO [StoreOpener-c8abf20a55c734610d779b0a9ffc4e0d-1] regionserver.HStore(310): Store=c8abf20a55c734610d779b0a9ffc4e0d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:53,183 INFO [StoreOpener-c8abf20a55c734610d779b0a9ffc4e0d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:53,184 DEBUG [StoreOpener-c8abf20a55c734610d779b0a9ffc4e0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d/cf2 2023-02-07 15:55:53,184 DEBUG [StoreOpener-c8abf20a55c734610d779b0a9ffc4e0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d/cf2 2023-02-07 15:55:53,185 INFO [StoreOpener-c8abf20a55c734610d779b0a9ffc4e0d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c8abf20a55c734610d779b0a9ffc4e0d columnFamilyName cf2 2023-02-07 15:55:53,185 INFO [StoreOpener-c8abf20a55c734610d779b0a9ffc4e0d-1] regionserver.HStore(310): Store=c8abf20a55c734610d779b0a9ffc4e0d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:53,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:53,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:53,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:55:53,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:53,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened c8abf20a55c734610d779b0a9ffc4e0d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62797474, jitterRate=-0.06424471735954285}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:55:53,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for c8abf20a55c734610d779b0a9ffc4e0d: Running coprocessor pre-open hook at 1675785353176Writing region info on filesystem at 1675785353176Initializing all the Stores at 1675785353177 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785353177Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785353178 (+1 ms)Cleaning up temporary data from old regions at 1675785353188 (+10 ms)Cleaning up detritus from prior splits at 1675785353188Running coprocessor post-open hooks at 1675785353191 (+3 ms)Region opened successfully at 1675785353191 2023-02-07 15:55:53,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d., pid=134, masterSystemTime=1675785353171 2023-02-07 15:55:53,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d. 2023-02-07 15:55:53,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d. 2023-02-07 15:55:53,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6. 2023-02-07 15:55:53,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5adddea27dcf0673e88a9fdaa5d479c6, NAME => 'testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.', STARTKEY => '', ENDKEY => '71'} 2023-02-07 15:55:53,194 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=c8abf20a55c734610d779b0a9ffc4e0d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:53,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:55:53,195 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675785353194"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785353194"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785353194"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785353194"}]},"ts":"1675785353194"} 2023-02-07 15:55:53,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 5adddea27dcf0673e88a9fdaa5d479c6 2023-02-07 15:55:53,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:53,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 5adddea27dcf0673e88a9fdaa5d479c6 2023-02-07 15:55:53,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 5adddea27dcf0673e88a9fdaa5d479c6 2023-02-07 15:55:53,196 INFO [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5adddea27dcf0673e88a9fdaa5d479c6 2023-02-07 15:55:53,197 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1 2023-02-07 15:55:53,197 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1 2023-02-07 15:55:53,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-02-07 15:55:53,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure c8abf20a55c734610d779b0a9ffc4e0d, server=jenkins-hbase12.apache.org,38629,1675785236797 in 173 msec 2023-02-07 15:55:53,197 INFO [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5adddea27dcf0673e88a9fdaa5d479c6 columnFamilyName cf1 2023-02-07 15:55:53,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c8abf20a55c734610d779b0a9ffc4e0d, ASSIGN in 333 msec 2023-02-07 15:55:53,201 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-0fe9f8e4d148440fb8004a2457a11ef8 2023-02-07 15:55:53,203 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-20dc002b6d6540cfb6253280b81f7e48 2023-02-07 15:55:53,206 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-4b18c9171b1440ab902e8888e7d1cfc9 2023-02-07 15:55:53,208 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-542298b43aad4e60948fa03f04b20f99 2023-02-07 15:55:53,211 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bb93ddbdbe4946caba401ad89fb1955c 2023-02-07 15:55:53,214 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bc26ea27b49b4b77a873fd630b8525ef 2023-02-07 15:55:53,216 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d0e0313700634e2b84a939e5a143b14f 2023-02-07 15:55:53,216 INFO [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(310): Store=5adddea27dcf0673e88a9fdaa5d479c6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:53,216 INFO [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5adddea27dcf0673e88a9fdaa5d479c6 2023-02-07 15:55:53,217 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2 2023-02-07 15:55:53,217 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2 2023-02-07 15:55:53,217 INFO [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5adddea27dcf0673e88a9fdaa5d479c6 columnFamilyName cf2 2023-02-07 15:55:53,221 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-1c1c2a3ee8f849e78f1ad3f402339a70 2023-02-07 15:55:53,223 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-5450b80b938d43f2b4b0d1ed2a1e176d 2023-02-07 15:55:53,225 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-99272da2a7ed4b0981b7c0aa89a2c020 2023-02-07 15:55:53,228 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-cffcf7e92a1e4fc0905a2fbea1e3faf4 2023-02-07 15:55:53,230 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d2e6b41e320f459c86bf67371ba0228e 2023-02-07 15:55:53,233 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-e3569c15560c4344b8aa7800047c2d55 2023-02-07 15:55:53,235 DEBUG [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-f5eb0bf8ca8f4f0797ee3456341f8d0d 2023-02-07 15:55:53,236 INFO [StoreOpener-5adddea27dcf0673e88a9fdaa5d479c6-1] regionserver.HStore(310): Store=5adddea27dcf0673e88a9fdaa5d479c6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:53,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6 2023-02-07 15:55:53,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6 2023-02-07 15:55:53,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:55:53,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 5adddea27dcf0673e88a9fdaa5d479c6 2023-02-07 15:55:53,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 5adddea27dcf0673e88a9fdaa5d479c6; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69798401, jitterRate=0.040077224373817444}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:55:53,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 5adddea27dcf0673e88a9fdaa5d479c6: Running coprocessor pre-open hook at 1675785353195Writing region info on filesystem at 1675785353195Initializing all the Stores at 1675785353196 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785353196Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785353196Cleaning up temporary data from old regions at 1675785353237 (+41 ms)Cleaning up detritus from prior splits at 1675785353237Running coprocessor post-open hooks at 1675785353239 (+2 ms)Region opened successfully at 1675785353239 2023-02-07 15:55:53,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6., pid=133, masterSystemTime=1675785353171 2023-02-07 15:55:53,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:53,240 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:55:53,240 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:55:53,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:55:53,240 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-07 15:55:53,241 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-07 15:55:53,241 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1912): 5adddea27dcf0673e88a9fdaa5d479c6/cf1 is initiating minor compaction (all files) 2023-02-07 15:55:53,241 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1912): 5adddea27dcf0673e88a9fdaa5d479c6/cf2 is initiating minor compaction (all files) 2023-02-07 15:55:53,241 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5adddea27dcf0673e88a9fdaa5d479c6/cf1 in testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6. 2023-02-07 15:55:53,241 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5adddea27dcf0673e88a9fdaa5d479c6/cf2 in testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6. 2023-02-07 15:55:53,241 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-4b18c9171b1440ab902e8888e7d1cfc9, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-542298b43aad4e60948fa03f04b20f99, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bc26ea27b49b4b77a873fd630b8525ef, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d0e0313700634e2b84a939e5a143b14f, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-20dc002b6d6540cfb6253280b81f7e48, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bb93ddbdbe4946caba401ad89fb1955c, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-0fe9f8e4d148440fb8004a2457a11ef8] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/.tmp, totalSize=34.9 K 2023-02-07 15:55:53,241 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-5450b80b938d43f2b4b0d1ed2a1e176d, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d2e6b41e320f459c86bf67371ba0228e, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-f5eb0bf8ca8f4f0797ee3456341f8d0d, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-99272da2a7ed4b0981b7c0aa89a2c020, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-cffcf7e92a1e4fc0905a2fbea1e3faf4, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-1c1c2a3ee8f849e78f1ad3f402339a70, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-e3569c15560c4344b8aa7800047c2d55] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/.tmp, totalSize=34.9 K 2023-02-07 15:55:53,241 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-4b18c9171b1440ab902e8888e7d1cfc9, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785349257 2023-02-07 15:55:53,241 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-5450b80b938d43f2b4b0d1ed2a1e176d, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785349257 2023-02-07 15:55:53,242 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-542298b43aad4e60948fa03f04b20f99, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785350132 2023-02-07 15:55:53,242 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d2e6b41e320f459c86bf67371ba0228e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785350132 2023-02-07 15:55:53,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6. 2023-02-07 15:55:53,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6. 2023-02-07 15:55:53,242 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bc26ea27b49b4b77a873fd630b8525ef, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785350393 2023-02-07 15:55:53,242 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-f5eb0bf8ca8f4f0797ee3456341f8d0d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785350393 2023-02-07 15:55:53,242 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=5adddea27dcf0673e88a9fdaa5d479c6, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:53,242 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d0e0313700634e2b84a939e5a143b14f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785351118 2023-02-07 15:55:53,242 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675785353242"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785353242"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785353242"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785353242"}]},"ts":"1675785353242"} 2023-02-07 15:55:53,242 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-99272da2a7ed4b0981b7c0aa89a2c020, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785351118 2023-02-07 15:55:53,242 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-20dc002b6d6540cfb6253280b81f7e48, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785351402 2023-02-07 15:55:53,242 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-cffcf7e92a1e4fc0905a2fbea1e3faf4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785351402 2023-02-07 15:55:53,243 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bb93ddbdbe4946caba401ad89fb1955c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785351652 2023-02-07 15:55:53,243 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-1c1c2a3ee8f849e78f1ad3f402339a70, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785351652 2023-02-07 15:55:53,243 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-0fe9f8e4d148440fb8004a2457a11ef8, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675785352126 2023-02-07 15:55:53,243 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-e3569c15560c4344b8aa7800047c2d55, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675785352126 2023-02-07 15:55:53,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-02-07 15:55:53,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure 5adddea27dcf0673e88a9fdaa5d479c6, server=jenkins-hbase12.apache.org,38629,1675785236797 in 225 msec 2023-02-07 15:55:53,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-02-07 15:55:53,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5adddea27dcf0673e88a9fdaa5d479c6, ASSIGN in 385 msec 2023-02-07 15:55:53,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=423dda2174efcce0795a0499add9f89e, daughterA=5adddea27dcf0673e88a9fdaa5d479c6, daughterB=c8abf20a55c734610d779b0a9ffc4e0d in 1.1110 sec 2023-02-07 15:55:53,261 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5adddea27dcf0673e88a9fdaa5d479c6#cf1#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:55:53,262 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] throttle.PressureAwareThroughputController(145): 5adddea27dcf0673e88a9fdaa5d479c6#cf2#compaction#127 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 15:55:53,270 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/.tmp/cf2/5fbeeb24b33c4492907b8648ed276a74 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/5fbeeb24b33c4492907b8648ed276a74 2023-02-07 15:55:53,270 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/.tmp/cf1/c4ff8a8fa8bd4a1f998e8f94c150fcce as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/c4ff8a8fa8bd4a1f998e8f94c150fcce 2023-02-07 15:55:53,274 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5adddea27dcf0673e88a9fdaa5d479c6/cf2 of 5adddea27dcf0673e88a9fdaa5d479c6 into 5fbeeb24b33c4492907b8648ed276a74(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:55:53,274 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5adddea27dcf0673e88a9fdaa5d479c6/cf1 of 5adddea27dcf0673e88a9fdaa5d479c6 into c4ff8a8fa8bd4a1f998e8f94c150fcce(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:55:53,274 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5adddea27dcf0673e88a9fdaa5d479c6: 2023-02-07 15:55:53,274 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5adddea27dcf0673e88a9fdaa5d479c6: 2023-02-07 15:55:53,274 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6., storeName=5adddea27dcf0673e88a9fdaa5d479c6/cf2, priority=9, startTime=1675785353240; duration=0sec 2023-02-07 15:55:53,274 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6., storeName=5adddea27dcf0673e88a9fdaa5d479c6/cf1, priority=9, startTime=1675785353240; duration=0sec 2023-02-07 15:55:53,274 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:53,274 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:53,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38629] ipc.CallRunner(144): callId: 995 service: ClientService methodName: Scan size: 157 connection: 136.243.104.168:51088 deadline: 1675785413391, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e. is not online on jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:53,502 DEBUG [Listener at localhost.localdomain/41965] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-07 15:55:53,507 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:55:53,507 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:53,507 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:53,508 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:53,508 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:53,509 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-07 15:55:53,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegionEmptyDaughter 2023-02-07 15:55:53,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 15:55:53,512 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:53,512 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:53,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-07 15:55:53,513 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785353513"}]},"ts":"1675785353513"} 2023-02-07 15:55:53,526 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-07 15:55:53,605 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-07 15:55:53,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5adddea27dcf0673e88a9fdaa5d479c6, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c8abf20a55c734610d779b0a9ffc4e0d, UNASSIGN}] 2023-02-07 15:55:53,607 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c8abf20a55c734610d779b0a9ffc4e0d, UNASSIGN 2023-02-07 15:55:53,607 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=c8abf20a55c734610d779b0a9ffc4e0d, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:53,608 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675785353607"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785353607"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785353607"}]},"ts":"1675785353607"} 2023-02-07 15:55:53,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure c8abf20a55c734610d779b0a9ffc4e0d, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:53,612 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5adddea27dcf0673e88a9fdaa5d479c6, UNASSIGN 2023-02-07 15:55:53,613 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=5adddea27dcf0673e88a9fdaa5d479c6, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:53,613 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675785353613"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785353613"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785353613"}]},"ts":"1675785353613"} 2023-02-07 15:55:53,613 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:53,613 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:53,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-07 15:55:53,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 5adddea27dcf0673e88a9fdaa5d479c6, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:53,627 ERROR [BP-1532654085-136.243.104.168-1675785230655 heartbeating to localhost.localdomain/127.0.0.1:43555] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1532654085-136.243.104.168-1675785230655 (Datanode Uuid 7f437b77-6c8e-4bcb-8556-91fa3c36506b) service to localhost.localdomain/127.0.0.1:43555 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 15:55:53,762 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:53,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing c8abf20a55c734610d779b0a9ffc4e0d, disabling compactions & flushes 2023-02-07 15:55:53,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d. 2023-02-07 15:55:53,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d. 2023-02-07 15:55:53,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d. after waiting 0 ms 2023-02-07 15:55:53,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d. 2023-02-07 15:55:53,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-07 15:55:53,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:55:53,767 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d. 2023-02-07 15:55:53,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for c8abf20a55c734610d779b0a9ffc4e0d: Waiting for close lock at 1675785353762Running coprocessor pre-close hooks at 1675785353762Disabling compacts and flushes for region at 1675785353762Disabling writes for close at 1675785353763 (+1 ms)Writing region close event to WAL at 1675785353763Running coprocessor post-close hooks at 1675785353767 (+4 ms)Closed at 1675785353767 2023-02-07 15:55:53,768 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:53,768 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 5adddea27dcf0673e88a9fdaa5d479c6 2023-02-07 15:55:53,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 5adddea27dcf0673e88a9fdaa5d479c6, disabling compactions & flushes 2023-02-07 15:55:53,768 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6. 2023-02-07 15:55:53,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6. 2023-02-07 15:55:53,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6. after waiting 0 ms 2023-02-07 15:55:53,768 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=c8abf20a55c734610d779b0a9ffc4e0d, regionState=CLOSED 2023-02-07 15:55:53,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6. 2023-02-07 15:55:53,768 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675785353768"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785353768"}]},"ts":"1675785353768"} 2023-02-07 15:55:53,769 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-4b18c9171b1440ab902e8888e7d1cfc9, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-542298b43aad4e60948fa03f04b20f99, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bc26ea27b49b4b77a873fd630b8525ef, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d0e0313700634e2b84a939e5a143b14f, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-20dc002b6d6540cfb6253280b81f7e48, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bb93ddbdbe4946caba401ad89fb1955c, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-0fe9f8e4d148440fb8004a2457a11ef8] to archive 2023-02-07 15:55:53,770 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:55:53,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-07 15:55:53,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure c8abf20a55c734610d779b0a9ffc4e0d, server=jenkins-hbase12.apache.org,38629,1675785236797 in 161 msec 2023-02-07 15:55:53,771 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-4b18c9171b1440ab902e8888e7d1cfc9 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-4b18c9171b1440ab902e8888e7d1cfc9 2023-02-07 15:55:53,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c8abf20a55c734610d779b0a9ffc4e0d, UNASSIGN in 165 msec 2023-02-07 15:55:53,772 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-542298b43aad4e60948fa03f04b20f99 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-542298b43aad4e60948fa03f04b20f99 2023-02-07 15:55:53,773 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bc26ea27b49b4b77a873fd630b8525ef to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bc26ea27b49b4b77a873fd630b8525ef 2023-02-07 15:55:53,774 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d0e0313700634e2b84a939e5a143b14f to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d0e0313700634e2b84a939e5a143b14f 2023-02-07 15:55:53,775 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-20dc002b6d6540cfb6253280b81f7e48 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-20dc002b6d6540cfb6253280b81f7e48 2023-02-07 15:55:53,775 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bb93ddbdbe4946caba401ad89fb1955c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-bb93ddbdbe4946caba401ad89fb1955c 2023-02-07 15:55:53,776 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-0fe9f8e4d148440fb8004a2457a11ef8 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-0fe9f8e4d148440fb8004a2457a11ef8 2023-02-07 15:55:53,778 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-5450b80b938d43f2b4b0d1ed2a1e176d, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d2e6b41e320f459c86bf67371ba0228e, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-f5eb0bf8ca8f4f0797ee3456341f8d0d, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-99272da2a7ed4b0981b7c0aa89a2c020, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-cffcf7e92a1e4fc0905a2fbea1e3faf4, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-1c1c2a3ee8f849e78f1ad3f402339a70, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-e3569c15560c4344b8aa7800047c2d55] to archive 2023-02-07 15:55:53,778 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:55:53,779 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-5450b80b938d43f2b4b0d1ed2a1e176d to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-5450b80b938d43f2b4b0d1ed2a1e176d 2023-02-07 15:55:53,780 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d2e6b41e320f459c86bf67371ba0228e to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-d2e6b41e320f459c86bf67371ba0228e 2023-02-07 15:55:53,781 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-f5eb0bf8ca8f4f0797ee3456341f8d0d to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-f5eb0bf8ca8f4f0797ee3456341f8d0d 2023-02-07 15:55:53,782 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-99272da2a7ed4b0981b7c0aa89a2c020 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-99272da2a7ed4b0981b7c0aa89a2c020 2023-02-07 15:55:53,783 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-cffcf7e92a1e4fc0905a2fbea1e3faf4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-cffcf7e92a1e4fc0905a2fbea1e3faf4 2023-02-07 15:55:53,784 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-1c1c2a3ee8f849e78f1ad3f402339a70 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-1c1c2a3ee8f849e78f1ad3f402339a70 2023-02-07 15:55:53,784 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-e3569c15560c4344b8aa7800047c2d55 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/testSplitTableRegionEmptyDaughter=423dda2174efcce0795a0499add9f89e-e3569c15560c4344b8aa7800047c2d55 2023-02-07 15:55:53,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 15:55:53,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:55:53,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6. 2023-02-07 15:55:53,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 5adddea27dcf0673e88a9fdaa5d479c6: Waiting for close lock at 1675785353768Running coprocessor pre-close hooks at 1675785353768Disabling compacts and flushes for region at 1675785353768Disabling writes for close at 1675785353768Writing region close event to WAL at 1675785353785 (+17 ms)Running coprocessor post-close hooks at 1675785353787 (+2 ms)Closed at 1675785353787 2023-02-07 15:55:53,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 5adddea27dcf0673e88a9fdaa5d479c6 2023-02-07 15:55:53,789 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=5adddea27dcf0673e88a9fdaa5d479c6, regionState=CLOSED 2023-02-07 15:55:53,789 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675785353789"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785353789"}]},"ts":"1675785353789"} 2023-02-07 15:55:53,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-07 15:55:53,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 5adddea27dcf0673e88a9fdaa5d479c6, server=jenkins-hbase12.apache.org,38629,1675785236797 in 167 msec 2023-02-07 15:55:53,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-02-07 15:55:53,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5adddea27dcf0673e88a9fdaa5d479c6, UNASSIGN in 185 msec 2023-02-07 15:55:53,792 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785353792"}]},"ts":"1675785353792"} 2023-02-07 15:55:53,793 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-07 15:55:53,805 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-07 15:55:53,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 296 msec 2023-02-07 15:55:53,814 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:53,815 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:53,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-07 15:55:53,816 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-07 15:55:53,816 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:53,816 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:53,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegionEmptyDaughter 2023-02-07 15:55:53,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 15:55:53,818 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 15:55:53,819 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:53,819 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:53,819 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 15:55:53,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-07 15:55:53,824 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:53,824 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:53,824 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6 2023-02-07 15:55:53,826 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d/recovered.edits] 2023-02-07 15:55:53,827 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/recovered.edits] 2023-02-07 15:55:53,827 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/recovered.edits] 2023-02-07 15:55:53,830 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/0fe9f8e4d148440fb8004a2457a11ef8 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/0fe9f8e4d148440fb8004a2457a11ef8 2023-02-07 15:55:53,830 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/c4ff8a8fa8bd4a1f998e8f94c150fcce to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf1/c4ff8a8fa8bd4a1f998e8f94c150fcce 2023-02-07 15:55:53,831 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/20dc002b6d6540cfb6253280b81f7e48 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/20dc002b6d6540cfb6253280b81f7e48 2023-02-07 15:55:53,832 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/4b18c9171b1440ab902e8888e7d1cfc9 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/4b18c9171b1440ab902e8888e7d1cfc9 2023-02-07 15:55:53,832 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d/recovered.edits/88.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d/recovered.edits/88.seqid 2023-02-07 15:55:53,832 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/5fbeeb24b33c4492907b8648ed276a74 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/cf2/5fbeeb24b33c4492907b8648ed276a74 2023-02-07 15:55:53,832 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/c8abf20a55c734610d779b0a9ffc4e0d 2023-02-07 15:55:53,833 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/542298b43aad4e60948fa03f04b20f99 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/542298b43aad4e60948fa03f04b20f99 2023-02-07 15:55:53,834 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bb93ddbdbe4946caba401ad89fb1955c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bb93ddbdbe4946caba401ad89fb1955c 2023-02-07 15:55:53,834 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/recovered.edits/90.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6/recovered.edits/90.seqid 2023-02-07 15:55:53,835 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/5adddea27dcf0673e88a9fdaa5d479c6 2023-02-07 15:55:53,835 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bc26ea27b49b4b77a873fd630b8525ef to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/bc26ea27b49b4b77a873fd630b8525ef 2023-02-07 15:55:53,836 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/d0e0313700634e2b84a939e5a143b14f to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf1/d0e0313700634e2b84a939e5a143b14f 2023-02-07 15:55:53,838 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/1c1c2a3ee8f849e78f1ad3f402339a70 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/1c1c2a3ee8f849e78f1ad3f402339a70 2023-02-07 15:55:53,839 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/5450b80b938d43f2b4b0d1ed2a1e176d to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/5450b80b938d43f2b4b0d1ed2a1e176d 2023-02-07 15:55:53,840 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/99272da2a7ed4b0981b7c0aa89a2c020 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/99272da2a7ed4b0981b7c0aa89a2c020 2023-02-07 15:55:53,841 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/cffcf7e92a1e4fc0905a2fbea1e3faf4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/cffcf7e92a1e4fc0905a2fbea1e3faf4 2023-02-07 15:55:53,841 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/d2e6b41e320f459c86bf67371ba0228e to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/d2e6b41e320f459c86bf67371ba0228e 2023-02-07 15:55:53,842 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/e3569c15560c4344b8aa7800047c2d55 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/e3569c15560c4344b8aa7800047c2d55 2023-02-07 15:55:53,843 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/f5eb0bf8ca8f4f0797ee3456341f8d0d to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/cf2/f5eb0bf8ca8f4f0797ee3456341f8d0d 2023-02-07 15:55:53,845 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/recovered.edits/85.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e/recovered.edits/85.seqid 2023-02-07 15:55:53,846 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegionEmptyDaughter/423dda2174efcce0795a0499add9f89e 2023-02-07 15:55:53,846 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-07 15:55:53,848 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 15:55:53,849 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-07 15:55:53,851 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-07 15:55:53,852 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 15:55:53,852 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-07 15:55:53,852 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785353852"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:53,852 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785353852"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:53,853 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785353852"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:53,854 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 15:55:53,854 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 423dda2174efcce0795a0499add9f89e, NAME => 'testSplitTableRegionEmptyDaughter,,1675785348753.423dda2174efcce0795a0499add9f89e.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 5adddea27dcf0673e88a9fdaa5d479c6, NAME => 'testSplitTableRegionEmptyDaughter,,1675785352139.5adddea27dcf0673e88a9fdaa5d479c6.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => c8abf20a55c734610d779b0a9ffc4e0d, NAME => 'testSplitTableRegionEmptyDaughter,71,1675785352139.c8abf20a55c734610d779b0a9ffc4e0d.', STARTKEY => '71', ENDKEY => ''}] 2023-02-07 15:55:53,854 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-07 15:55:53,854 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675785353854"}]},"ts":"9223372036854775807"} 2023-02-07 15:55:53,856 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-07 15:55:53,868 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 15:55:53,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 52 msec 2023-02-07 15:55:53,920 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:53,920 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:53,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-07 15:55:53,921 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-07 15:55:53,936 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=374 (was 374), OpenFileDescriptor=847 (was 847), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=173 (was 188), ProcessCount=174 (was 174), AvailableMemoryMB=2909 (was 3014) 2023-02-07 15:55:53,950 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=374, OpenFileDescriptor=847, MaxFileDescriptor=60000, SystemLoadAverage=173, ProcessCount=174, AvailableMemoryMB=2908 2023-02-07 15:55:53,950 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:55:53,950 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:53,951 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:53,962 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 15:55:53,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 15:55:53,964 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-07 15:55:53,967 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 15:55:53,967 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 15:55:53,971 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:53,972 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4 empty. 2023-02-07 15:55:53,974 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:53,974 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-07 15:55:54,389 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-07 15:55:54,392 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1ef1944f776e9a061c872d33197334a4, NAME => 'testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp 2023-02-07 15:55:54,407 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:54,407 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 1ef1944f776e9a061c872d33197334a4, disabling compactions & flushes 2023-02-07 15:55:54,407 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:54,407 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:54,407 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. after waiting 0 ms 2023-02-07 15:55:54,407 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:54,408 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:54,408 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 1ef1944f776e9a061c872d33197334a4: Waiting for close lock at 1675785354407Disabling compacts and flushes for region at 1675785354407Disabling writes for close at 1675785354407Writing region close event to WAL at 1675785354408 (+1 ms)Closed at 1675785354408 2023-02-07 15:55:54,410 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 15:55:54,411 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785354411"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785354411"}]},"ts":"1675785354411"} 2023-02-07 15:55:54,412 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 15:55:54,413 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 15:55:54,413 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785354413"}]},"ts":"1675785354413"} 2023-02-07 15:55:54,414 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-07 15:55:54,479 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 15:55:54,481 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 15:55:54,481 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 15:55:54,482 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 15:55:54,482 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 15:55:54,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=1ef1944f776e9a061c872d33197334a4, ASSIGN}] 2023-02-07 15:55:54,484 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=1ef1944f776e9a061c872d33197334a4, ASSIGN 2023-02-07 15:55:54,485 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=1ef1944f776e9a061c872d33197334a4, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:55:54,626 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 15:55:54,635 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 15:55:54,636 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=1ef1944f776e9a061c872d33197334a4, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:54,636 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785354636"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785354636"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785354636"}]},"ts":"1675785354636"} 2023-02-07 15:55:54,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 1ef1944f776e9a061c872d33197334a4, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:54,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:54,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1ef1944f776e9a061c872d33197334a4, NAME => 'testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4.', STARTKEY => '', ENDKEY => ''} 2023-02-07 15:55:54,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:55:54,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:54,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:54,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:54,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:54,799 INFO [StoreOpener-1ef1944f776e9a061c872d33197334a4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:54,801 DEBUG [StoreOpener-1ef1944f776e9a061c872d33197334a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1 2023-02-07 15:55:54,801 DEBUG [StoreOpener-1ef1944f776e9a061c872d33197334a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1 2023-02-07 15:55:54,802 INFO [StoreOpener-1ef1944f776e9a061c872d33197334a4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ef1944f776e9a061c872d33197334a4 columnFamilyName cf1 2023-02-07 15:55:54,802 INFO [StoreOpener-1ef1944f776e9a061c872d33197334a4-1] regionserver.HStore(310): Store=1ef1944f776e9a061c872d33197334a4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:54,803 INFO [StoreOpener-1ef1944f776e9a061c872d33197334a4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:54,804 DEBUG [StoreOpener-1ef1944f776e9a061c872d33197334a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2 2023-02-07 15:55:54,804 DEBUG [StoreOpener-1ef1944f776e9a061c872d33197334a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2 2023-02-07 15:55:54,805 INFO [StoreOpener-1ef1944f776e9a061c872d33197334a4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ef1944f776e9a061c872d33197334a4 columnFamilyName cf2 2023-02-07 15:55:54,805 INFO [StoreOpener-1ef1944f776e9a061c872d33197334a4-1] regionserver.HStore(310): Store=1ef1944f776e9a061c872d33197334a4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:54,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:54,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:54,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:55:54,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:54,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 15:55:54,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 1ef1944f776e9a061c872d33197334a4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61589433, jitterRate=-0.08224593102931976}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:55:54,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 1ef1944f776e9a061c872d33197334a4: Running coprocessor pre-open hook at 1675785354797Writing region info on filesystem at 1675785354797Initializing all the Stores at 1675785354799 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785354799Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785354799Cleaning up temporary data from old regions at 1675785354807 (+8 ms)Cleaning up detritus from prior splits at 1675785354808 (+1 ms)Running coprocessor post-open hooks at 1675785354812 (+4 ms)Region opened successfully at 1675785354812 2023-02-07 15:55:54,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4., pid=143, masterSystemTime=1675785354789 2023-02-07 15:55:54,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:54,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:54,814 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=1ef1944f776e9a061c872d33197334a4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:54,814 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785354814"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785354814"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785354814"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785354814"}]},"ts":"1675785354814"} 2023-02-07 15:55:54,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-07 15:55:54,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 1ef1944f776e9a061c872d33197334a4, server=jenkins-hbase12.apache.org,38629,1675785236797 in 178 msec 2023-02-07 15:55:54,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-07 15:55:54,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=1ef1944f776e9a061c872d33197334a4, ASSIGN in 334 msec 2023-02-07 15:55:54,818 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 15:55:54,818 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785354818"}]},"ts":"1675785354818"} 2023-02-07 15:55:54,819 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-07 15:55:54,826 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 15:55:54,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 863 msec 2023-02-07 15:55:54,979 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:54,980 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:54,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:54,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-07 15:55:54,982 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-07 15:55:54,982 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:54,982 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-07 15:55:54,982 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-07 15:55:54,983 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:54,983 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:54,994 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:54,994 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:54,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:54,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:54,994 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:54,994 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:54,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:54,995 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:54,995 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:54,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:54,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:54,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:54,996 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:54,996 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:54,996 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:54,996 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:54,996 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:54,996 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:54,996 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:54,996 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:54,996 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:54,996 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:54,997 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:54,997 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:54,997 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:54,997 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:54,997 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:54,997 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:54,998 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:54,998 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:54,998 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:54,998 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. started... 2023-02-07 15:55:54,999 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1ef1944f776e9a061c872d33197334a4 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 15:55:55,004 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,009 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/84fc4227281545bbb7ee3351320e43f9 2023-02-07 15:55:55,015 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,015 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,015 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:55,016 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,016 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:55,024 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec 2023-02-07 15:55:55,028 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/84fc4227281545bbb7ee3351320e43f9 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/84fc4227281545bbb7ee3351320e43f9 2023-02-07 15:55:55,032 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/84fc4227281545bbb7ee3351320e43f9, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:55:55,033 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec 2023-02-07 15:55:55,036 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:55:55,037 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 1ef1944f776e9a061c872d33197334a4 in 38ms, sequenceid=6, compaction requested=false 2023-02-07 15:55:55,037 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-07 15:55:55,038 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1ef1944f776e9a061c872d33197334a4: 2023-02-07 15:55:55,038 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:55,038 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:55,038 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:55,038 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,038 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:55,038 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:55,047 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,047 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:55,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:55,047 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,047 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:55,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:55,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:55,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:55,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-07 15:55:55,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9b4c823[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:55,052 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-07 15:55:55,052 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,106 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,106 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,106 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,107 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:55,107 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,107 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:55,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,107 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:55,108 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:55,108 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:55,107 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:55,108 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:55,108 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:55,109 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:55,109 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:55,131 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:55,131 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,131 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:55,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:55,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:55,132 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:55,142 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:55,142 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:55,142 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:55,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:55,142 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:55,143 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:55,143 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:55,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:55,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:55,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-07 15:55:55,153 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-07 15:55:55,153 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:55,153 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:55,153 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-07 15:55:55,153 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:55,163 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,163 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,163 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:55,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:55,163 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,163 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,163 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:55,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,163 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,164 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,165 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,166 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,170 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,171 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,171 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,184 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:55,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-07 15:55:55,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:55,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,185 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:55,185 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:55,184 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,185 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,185 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,185 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,185 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,186 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,285 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:55,285 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:55,286 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:55,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:55,309 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:55,309 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:55,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:55,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:55,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-07 15:55:55,312 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-07 15:55:55,312 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:55,312 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-07 15:55:55,312 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-07 15:55:55,313 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,313 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,320 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,320 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,320 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,320 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,321 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,321 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,323 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,323 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:55,323 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:55,323 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:55,323 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:55,323 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:55,323 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:55,323 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:55,324 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:55,324 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:55,324 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:55,324 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:55,324 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:55,324 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:55,324 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:55,324 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:55,324 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:55,324 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:55,324 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:55,324 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:55,325 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:55,324 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:55,324 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:55,325 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:55,325 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:55,325 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. started... 2023-02-07 15:55:55,325 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1ef1944f776e9a061c872d33197334a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:55,331 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,340 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c 2023-02-07 15:55:55,341 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,341 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,341 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:55,342 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,342 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:55,349 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/54fdf8c1c41a482992c6b9e8fda7b506 2023-02-07 15:55:55,353 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c 2023-02-07 15:55:55,356 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:55:55,357 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/54fdf8c1c41a482992c6b9e8fda7b506 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/54fdf8c1c41a482992c6b9e8fda7b506 2023-02-07 15:55:55,360 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/54fdf8c1c41a482992c6b9e8fda7b506, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 15:55:55,361 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1ef1944f776e9a061c872d33197334a4 in 36ms, sequenceid=19, compaction requested=false 2023-02-07 15:55:55,361 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1ef1944f776e9a061c872d33197334a4: 2023-02-07 15:55:55,361 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:55,361 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:55,361 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:55,361 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,361 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:55,361 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:55,373 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,373 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:55,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:55,373 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,374 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:55,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:55,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:55,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:55,376 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-07 15:55:55,376 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6312378a[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:55,376 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-07 15:55:55,376 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,384 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,384 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,384 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,384 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,384 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:55,384 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:55,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,384 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:55,384 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:55,384 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:55,385 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:55,384 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:55,385 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:55,385 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:55,385 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:55,394 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:55,394 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:55,394 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:55,394 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:55,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:55,405 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:55,405 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:55,405 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:55,405 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:55,405 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:55,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:55,405 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:55,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:55,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:55,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:55,415 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-07 15:55:55,415 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-07 15:55:55,415 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:55,415 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:55,415 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-07 15:55:55,415 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:55,426 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,426 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,426 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,426 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,426 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,426 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,427 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,427 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:55,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:55,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,429 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:55,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,430 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:55,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,434 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:55,435 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,436 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,476 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,476 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,476 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,476 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-07 15:55:55,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:55,476 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,477 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,477 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,477 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,477 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:55,477 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:55,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,476 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,477 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:55,477 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,477 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,479 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,480 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,480 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,480 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,480 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,480 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,579 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:55,579 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:55,579 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:55,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:55,596 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:55,596 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:55,597 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:55,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:55,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-07 15:55:55,598 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-07 15:55:55,598 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:55,598 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-07 15:55:55,598 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-07 15:55:55,599 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,599 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,615 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,615 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,615 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,615 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,616 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,616 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:55,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,617 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:55,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,617 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,617 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:55,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:55,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,617 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:55,617 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:55,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:55,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:55,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:55,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:55,618 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:55,618 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:55,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:55,618 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:55,618 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:55,618 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:55,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:55,618 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:55,618 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:55,618 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:55,618 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:55,618 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. started... 2023-02-07 15:55:55,618 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:55,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:55,618 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:55,619 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:55,619 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:55,619 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1ef1944f776e9a061c872d33197334a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:55,619 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:55,619 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:55,619 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:55,626 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,629 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/395de32f28b64f6f8705ee57a236e690 2023-02-07 15:55:55,636 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,636 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,636 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:55,636 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,636 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:55,638 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/0722de5250584b3abf459a4c95bf348b 2023-02-07 15:55:55,641 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/395de32f28b64f6f8705ee57a236e690 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/395de32f28b64f6f8705ee57a236e690 2023-02-07 15:55:55,644 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/395de32f28b64f6f8705ee57a236e690, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:55:55,644 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/0722de5250584b3abf459a4c95bf348b as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/0722de5250584b3abf459a4c95bf348b 2023-02-07 15:55:55,646 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/0722de5250584b3abf459a4c95bf348b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 15:55:55,647 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1ef1944f776e9a061c872d33197334a4 in 28ms, sequenceid=32, compaction requested=true 2023-02-07 15:55:55,647 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1ef1944f776e9a061c872d33197334a4: 2023-02-07 15:55:55,647 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:55,647 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:55,647 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:55,647 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,647 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:55,647 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:55,657 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,657 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:55,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:55,658 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,658 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:55,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:55,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:55,660 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-07 15:55:55,660 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71f1e997[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:55,660 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-07 15:55:55,660 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,668 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,668 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,668 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,668 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:55,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,668 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,668 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:55,668 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:55,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,669 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:55,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,669 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:55,669 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:55,669 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:55,669 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:55,669 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:55,669 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:55,678 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:55,678 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:55,678 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:55,689 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:55,689 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,689 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:55,689 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:55,689 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:55,690 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:55,689 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:55,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:55,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:55,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:55,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:55,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:55,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-07 15:55:55,701 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-07 15:55:55,701 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:55,701 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:55,701 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-07 15:55:55,702 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:55,716 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,716 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,716 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,716 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,716 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,716 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,716 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,716 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:55,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:55,716 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:55,717 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:55,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,724 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,724 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:55,724 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:55,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,731 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,731 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,731 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,731 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,731 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:55,731 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,731 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,731 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-07 15:55:55,731 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:55,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,731 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,731 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:55,731 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:55,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:55,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,732 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:55,731 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:55,732 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,732 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,732 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:55,732 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,732 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:55,732 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,732 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,732 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,832 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:55,832 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:55,832 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:55,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:55,843 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:55,844 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:55,844 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:55,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:55,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-07 15:55:55,848 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-07 15:55:55,848 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:55,848 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-07 15:55:55,848 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-07 15:55:55,848 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,848 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,857 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,857 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,857 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:55,857 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:55,858 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:55,858 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:55,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:55,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:55,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:55,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:55,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:55,859 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:55,859 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:55,859 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:55,859 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:55,859 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:55,859 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:55,859 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:55,859 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:55,859 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. started... 2023-02-07 15:55:55,860 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1ef1944f776e9a061c872d33197334a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:55,868 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,868 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,868 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,868 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:55,868 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:55,868 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:55,868 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/d3107a7f511843f0af863b0bc529c1bd 2023-02-07 15:55:56,286 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/6c57e193e0ed4d5d80dec167921e4dfd 2023-02-07 15:55:56,296 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/d3107a7f511843f0af863b0bc529c1bd as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/d3107a7f511843f0af863b0bc529c1bd 2023-02-07 15:55:56,304 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/d3107a7f511843f0af863b0bc529c1bd, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:55:56,305 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/6c57e193e0ed4d5d80dec167921e4dfd as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/6c57e193e0ed4d5d80dec167921e4dfd 2023-02-07 15:55:56,310 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/6c57e193e0ed4d5d80dec167921e4dfd, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 15:55:56,311 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1ef1944f776e9a061c872d33197334a4 in 451ms, sequenceid=45, compaction requested=true 2023-02-07 15:55:56,311 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1ef1944f776e9a061c872d33197334a4: 2023-02-07 15:55:56,311 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:56,311 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:56,311 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:56,311 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,311 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:56,311 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:56,363 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,363 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:56,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:56,364 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,364 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:56,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:56,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:56,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:56,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:56,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:56,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:56,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-07 15:55:56,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c698a8e[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:56,371 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-07 15:55:56,372 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,383 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,383 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,383 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,384 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:56,384 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:56,384 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:56,384 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:56,384 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:56,384 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:56,385 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:56,385 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:56,384 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,385 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:56,385 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:56,394 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:56,394 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:56,394 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,394 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:56,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:56,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:56,405 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:56,405 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:56,405 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:56,446 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:56,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:56,446 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:56,446 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:56,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:56,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:56,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:56,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:56,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:56,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:56,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:56,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:56,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:56,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-07 15:55:56,460 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-07 15:55:56,460 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:56,460 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:56,460 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-07 15:55:56,460 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:56,479 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,479 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:56,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:56,479 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,479 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,479 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:56,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:56,479 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:56,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:56,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:56,480 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:56,480 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:56,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:56,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:56,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,482 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:56,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,483 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:56,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:56,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:56,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:56,486 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:56,486 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:56,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,487 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:56,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:56,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:56,499 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:56,499 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:56,499 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:56,499 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:56,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:56,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-07 15:55:56,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:56,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:56,500 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:56,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:56,499 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:56,499 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:56,499 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:56,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:56,499 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:56,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:56,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:56,500 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:56,500 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:56,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:56,500 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:56,500 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:56,500 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:56,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:56,500 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:56,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:56,500 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:56,501 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:56,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:56,501 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,501 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,501 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,600 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:56,601 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:56,601 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:56,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:56,616 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:56,616 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:56,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:56,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:56,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-07 15:55:56,618 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-07 15:55:56,618 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:56,618 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-07 15:55:56,618 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-07 15:55:56,619 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,619 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:56,626 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:56,626 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,626 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:56,626 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:56,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:56,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:56,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:56,626 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:56,626 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:56,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:56,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:56,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:56,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:56,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:56,627 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:56,628 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:56,628 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:56,628 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:56,628 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:56,628 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:56,628 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:56,628 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:56,628 ERROR [BP-1532654085-136.243.104.168-1675785230655 heartbeating to localhost.localdomain/127.0.0.1:43555] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1532654085-136.243.104.168-1675785230655 (Datanode Uuid bb8a498b-c978-4fc3-b7cd-17da28adf753) service to localhost.localdomain/127.0.0.1:43555 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 15:55:56,628 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:56,628 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:56,628 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:56,628 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:56,628 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:56,628 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:56,628 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:56,628 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:56,628 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:56,628 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:56,629 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:56,628 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:56,629 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:56,629 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:56,629 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:56,629 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. started... 2023-02-07 15:55:56,629 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1ef1944f776e9a061c872d33197334a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:56,636 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,647 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,647 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,647 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:56,647 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:56,647 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:57,843 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/8358a309ce184e3598635c25e33805a2 2023-02-07 15:55:57,864 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/fa6b32b46e7c461ca8db86ad0898239b 2023-02-07 15:55:57,869 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/8358a309ce184e3598635c25e33805a2 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/8358a309ce184e3598635c25e33805a2 2023-02-07 15:55:57,872 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/8358a309ce184e3598635c25e33805a2, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:55:57,872 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/fa6b32b46e7c461ca8db86ad0898239b as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/fa6b32b46e7c461ca8db86ad0898239b 2023-02-07 15:55:57,875 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/fa6b32b46e7c461ca8db86ad0898239b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 15:55:57,875 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1ef1944f776e9a061c872d33197334a4 in 1246ms, sequenceid=58, compaction requested=true 2023-02-07 15:55:57,876 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1ef1944f776e9a061c872d33197334a4: 2023-02-07 15:55:57,876 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:57,876 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:57,876 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:57,876 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,876 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:57,876 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:57,886 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,886 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:57,886 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,886 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:57,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:57,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:57,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:57,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:57,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:57,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-07 15:55:57,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5333b0f0[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:57,889 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-07 15:55:57,889 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,899 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,899 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,899 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,900 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:57,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,900 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:57,900 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:57,900 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:57,900 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:57,900 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,900 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:57,900 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:57,900 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:57,900 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:57,900 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:57,910 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:57,910 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:57,910 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:57,920 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:57,920 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:57,920 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:57,921 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:57,920 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,921 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:57,921 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:57,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:57,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:57,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:57,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:57,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:57,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:57,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:57,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:57,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:57,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:57,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-07 15:55:57,933 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-07 15:55:57,933 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:57,933 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:57,933 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-07 15:55:57,933 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:57,941 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,941 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,941 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,941 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:57,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:57,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,942 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:57,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,941 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:57,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,941 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:57,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:57,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:57,943 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:57,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:57,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:57,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:57,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:57,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:57,943 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:57,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:57,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:57,944 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:57,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:57,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:57,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:57,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:57,946 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:57,946 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:57,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,947 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:57,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:57,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:57,962 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:57,962 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:57,962 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:57,962 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:57,962 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:57,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-07 15:55:57,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:57,962 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:57,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:57,962 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:57,962 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,963 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:57,962 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:57,963 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:57,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:57,963 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:57,963 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:57,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:57,963 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:57,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:57,963 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:57,963 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:57,963 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:57,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:57,963 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:57,963 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:57,963 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,063 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:58,063 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:58,063 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:58,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:58,073 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:58,073 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:58,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 15:55:58,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-07 15:55:58,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-07 15:55:58,075 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-07 15:55:58,075 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:58,075 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-07 15:55:58,075 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-07 15:55:58,075 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,075 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:58,086 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:58,086 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:58,086 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:58,086 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:58,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:58,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:58,086 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,086 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 15:55:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:58,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 15:55:58,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:58,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:58,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,35657,1675785236854' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:58,087 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37399,1675785236898' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:58,087 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 15:55:58,088 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 15:55:58,088 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 15:55:58,088 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:58,088 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. started... 2023-02-07 15:55:58,088 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1ef1944f776e9a061c872d33197334a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 15:55:58,096 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/bd5295dc14b04b63b78dcf385fd42064 2023-02-07 15:55:58,106 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/185de16e817e4ae7b80491991f2af9e5 2023-02-07 15:55:58,109 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/bd5295dc14b04b63b78dcf385fd42064 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/bd5295dc14b04b63b78dcf385fd42064 2023-02-07 15:55:58,112 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/bd5295dc14b04b63b78dcf385fd42064, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:55:58,113 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/185de16e817e4ae7b80491991f2af9e5 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/185de16e817e4ae7b80491991f2af9e5 2023-02-07 15:55:58,116 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/185de16e817e4ae7b80491991f2af9e5, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 15:55:58,117 INFO [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1ef1944f776e9a061c872d33197334a4 in 29ms, sequenceid=71, compaction requested=true 2023-02-07 15:55:58,117 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1ef1944f776e9a061c872d33197334a4: 2023-02-07 15:55:58,117 DEBUG [rs(jenkins-hbase12.apache.org,38629,1675785236797)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:58,117 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 15:55:58,117 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 15:55:58,117 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,117 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 15:55:58,117 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 15:55:58,131 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,132 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,132 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:58,152 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,152 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,152 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:58,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:58,152 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,152 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,152 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:58,152 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 15:55:58,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:58,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:58,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:58,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:58,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:58,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:58,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,38629,1675785236797' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-07 15:55:58,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@216f737b[Count = 0] remaining members to acquire global barrier 2023-02-07 15:55:58,154 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-07 15:55:58,154 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,162 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,162 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,162 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,163 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,163 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:58,163 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:58,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,163 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:58,163 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:58,163 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,38629,1675785236797' in zk 2023-02-07 15:55:58,163 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,35657,1675785236854' in zk 2023-02-07 15:55:58,163 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 15:55:58,163 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 15:55:58,163 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 15:55:58,163 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,37399,1675785236898' in zk 2023-02-07 15:55:58,173 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:58,173 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,173 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:58,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:58,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:58,173 DEBUG [member: 'jenkins-hbase12.apache.org,38629,1675785236797' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:58,183 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:58,183 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:58,183 DEBUG [member: 'jenkins-hbase12.apache.org,35657,1675785236854' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:58,194 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 15:55:58,194 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:58,194 DEBUG [member: 'jenkins-hbase12.apache.org,37399,1675785236898' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 15:55:58,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:58,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:58,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:58,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:58,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:58,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:58,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:58,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:58,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:58,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,38629,1675785236797': 2023-02-07 15:55:58,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,38629,1675785236797' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-07 15:55:58,197 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-07 15:55:58,197 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 15:55:58,197 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 15:55:58,197 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-07 15:55:58,197 INFO [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 15:55:58,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 15:55:58,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:58,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,205 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:58,204 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:58,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 15:55:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,205 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 15:55:58,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:58,205 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:58,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:58,205 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 15:55:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:58,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:58,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:58,206 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 15:55:58,207 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 15:55:58,207 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:58,215 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:58,215 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:58,215 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:58,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:58,215 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:58,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-07 15:55:58,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 15:55:58,215 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:58,215 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:58,215 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 15:55:58,215 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-07 15:55:58,215 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 15:55:58,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:58,215 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:58,215 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,215 DEBUG [(jenkins-hbase12.apache.org,37073,1675785235583)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 15:55:58,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:58,216 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 15:55:58,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:58,215 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 15:55:58,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 15:55:58,216 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:55:58,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 15:55:58,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:58,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:58,216 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:58,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:58,216 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:55:58,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 15:55:58,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 15:55:58,216 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,216 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 15:55:58,216 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 15:55:58,315 DEBUG [Listener at localhost.localdomain/41965] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 15:55:58,316 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:58,316 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:58,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 15:55:58,323 DEBUG [Listener at localhost.localdomain/41965] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,323 DEBUG [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=1ef1944f776e9a061c872d33197334a4, daughterA=db7af127455310d075d9f48aa368dca1, daughterB=f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:58,324 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=1ef1944f776e9a061c872d33197334a4, daughterA=db7af127455310d075d9f48aa368dca1, daughterB=f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:58,324 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=1ef1944f776e9a061c872d33197334a4, daughterA=db7af127455310d075d9f48aa368dca1, daughterB=f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:58,324 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=1ef1944f776e9a061c872d33197334a4, daughterA=db7af127455310d075d9f48aa368dca1, daughterB=f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:58,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=1ef1944f776e9a061c872d33197334a4, UNASSIGN}] 2023-02-07 15:55:58,326 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=1ef1944f776e9a061c872d33197334a4, UNASSIGN 2023-02-07 15:55:58,327 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=1ef1944f776e9a061c872d33197334a4, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:58,327 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785358327"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785358327"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785358327"}]},"ts":"1675785358327"} 2023-02-07 15:55:58,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 1ef1944f776e9a061c872d33197334a4, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:58,406 DEBUG [regionserver/jenkins-hbase12:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-07 15:55:58,414 DEBUG [regionserver/jenkins-hbase12:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-07 15:55:58,479 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,479 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 1ef1944f776e9a061c872d33197334a4, disabling compactions & flushes 2023-02-07 15:55:58,479 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:58,479 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:58,479 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. after waiting 0 ms 2023-02-07 15:55:58,479 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:58,479 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 1ef1944f776e9a061c872d33197334a4 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 15:55:58,488 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445 2023-02-07 15:55:58,498 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/5b70123e69544a2cb22394b029386535 2023-02-07 15:55:58,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445 2023-02-07 15:55:58,504 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:55:58,505 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/.tmp/cf2/5b70123e69544a2cb22394b029386535 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/5b70123e69544a2cb22394b029386535 2023-02-07 15:55:58,508 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/5b70123e69544a2cb22394b029386535, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 15:55:58,509 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1ef1944f776e9a061c872d33197334a4 in 29ms, sequenceid=82, compaction requested=true 2023-02-07 15:55:58,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 15:55:58,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:55:58,512 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. 2023-02-07 15:55:58,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 1ef1944f776e9a061c872d33197334a4: Waiting for close lock at 1675785358479Running coprocessor pre-close hooks at 1675785358479Disabling compacts and flushes for region at 1675785358479Disabling writes for close at 1675785358479Obtaining lock to block concurrent updates at 1675785358479Preparing flush snapshotting stores in 1ef1944f776e9a061c872d33197334a4 at 1675785358479Finished memstore snapshotting testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675785358480 (+1 ms)Flushing stores of testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. at 1675785358480Flushing 1ef1944f776e9a061c872d33197334a4/cf1: creating writer at 1675785358481 (+1 ms)Flushing 1ef1944f776e9a061c872d33197334a4/cf1: appending metadata at 1675785358483 (+2 ms)Flushing 1ef1944f776e9a061c872d33197334a4/cf1: closing flushed file at 1675785358483Flushing 1ef1944f776e9a061c872d33197334a4/cf2: creating writer at 1675785358491 (+8 ms)Flushing 1ef1944f776e9a061c872d33197334a4/cf2: appending metadata at 1675785358492 (+1 ms)Flushing 1ef1944f776e9a061c872d33197334a4/cf2: closing flushed file at 1675785358492Flushing 1ef1944f776e9a061c872d33197334a4/cf1: reopening flushed file at 1675785358502 (+10 ms)Flushing 1ef1944f776e9a061c872d33197334a4/cf2: reopening flushed file at 1675785358505 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1ef1944f776e9a061c872d33197334a4 in 29ms, sequenceid=82, compaction requested=true at 1675785358509 (+4 ms)Writing region close event to WAL at 1675785358510 (+1 ms)Running coprocessor post-close hooks at 1675785358512 (+2 ms)Closed at 1675785358512 2023-02-07 15:55:58,513 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,513 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=1ef1944f776e9a061c872d33197334a4, regionState=CLOSED 2023-02-07 15:55:58,513 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785358513"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785358513"}]},"ts":"1675785358513"} 2023-02-07 15:55:58,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-07 15:55:58,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 1ef1944f776e9a061c872d33197334a4, server=jenkins-hbase12.apache.org,38629,1675785236797 in 186 msec 2023-02-07 15:55:58,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-07 15:55:58,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=1ef1944f776e9a061c872d33197334a4, UNASSIGN in 189 msec 2023-02-07 15:55:58,524 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=1ef1944f776e9a061c872d33197334a4, threads=14 2023-02-07 15:55:58,524 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/0722de5250584b3abf459a4c95bf348b for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,524 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/185de16e817e4ae7b80491991f2af9e5 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,524 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/54fdf8c1c41a482992c6b9e8fda7b506 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,524 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/5b70123e69544a2cb22394b029386535 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,524 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/6c57e193e0ed4d5d80dec167921e4dfd for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,524 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,525 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,525 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/395de32f28b64f6f8705ee57a236e690 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,525 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/fa6b32b46e7c461ca8db86ad0898239b for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,525 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/8358a309ce184e3598635c25e33805a2 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,526 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/84fc4227281545bbb7ee3351320e43f9 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,526 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/d3107a7f511843f0af863b0bc529c1bd for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,526 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/bd5295dc14b04b63b78dcf385fd42064 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,529 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,650 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/6c57e193e0ed4d5d80dec167921e4dfd for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,653 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/bd5295dc14b04b63b78dcf385fd42064 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,673 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/d3107a7f511843f0af863b0bc529c1bd for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,673 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/5b70123e69544a2cb22394b029386535 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,680 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/395de32f28b64f6f8705ee57a236e690 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,685 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/185de16e817e4ae7b80491991f2af9e5 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,693 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,697 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/fa6b32b46e7c461ca8db86ad0898239b for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,697 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/84fc4227281545bbb7ee3351320e43f9 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,698 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,698 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/54fdf8c1c41a482992c6b9e8fda7b506 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,700 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:58,700 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/8358a309ce184e3598635c25e33805a2 for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,056 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/0722de5250584b3abf459a4c95bf348b for region: 1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,056 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 1ef1944f776e9a061c872d33197334a4 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 15:55:59,092 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 15:55:59,093 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 15:55:59,094 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675785359094"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675785359094"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675785359094"}]},"ts":"1675785359094"} 2023-02-07 15:55:59,094 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785359094"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785359094"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785359094"}]},"ts":"1675785359094"} 2023-02-07 15:55:59,095 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785359094"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785359094"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785359094"}]},"ts":"1675785359094"} 2023-02-07 15:55:59,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=db7af127455310d075d9f48aa368dca1, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=f10c2c4ee9fd52ac49d41344e75d0dd4, ASSIGN}] 2023-02-07 15:55:59,097 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=f10c2c4ee9fd52ac49d41344e75d0dd4, ASSIGN 2023-02-07 15:55:59,098 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=f10c2c4ee9fd52ac49d41344e75d0dd4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:55:59,098 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=db7af127455310d075d9f48aa368dca1, ASSIGN 2023-02-07 15:55:59,098 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=db7af127455310d075d9f48aa368dca1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,38629,1675785236797; forceNewPlan=false, retain=false 2023-02-07 15:55:59,248 INFO [jenkins-hbase12:37073] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 15:55:59,250 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=db7af127455310d075d9f48aa368dca1, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:59,251 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785359250"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785359250"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785359250"}]},"ts":"1675785359250"} 2023-02-07 15:55:59,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure db7af127455310d075d9f48aa368dca1, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:59,257 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=f10c2c4ee9fd52ac49d41344e75d0dd4, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:59,257 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785359257"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785359257"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785359257"}]},"ts":"1675785359257"} 2023-02-07 15:55:59,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure f10c2c4ee9fd52ac49d41344e75d0dd4, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:59,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1. 2023-02-07 15:55:59,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => db7af127455310d075d9f48aa368dca1, NAME => 'testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.', STARTKEY => '', ENDKEY => '41'} 2023-02-07 15:55:59,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:55:59,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion db7af127455310d075d9f48aa368dca1 2023-02-07 15:55:59,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:59,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for db7af127455310d075d9f48aa368dca1 2023-02-07 15:55:59,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for db7af127455310d075d9f48aa368dca1 2023-02-07 15:55:59,411 INFO [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region db7af127455310d075d9f48aa368dca1 2023-02-07 15:55:59,412 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1 2023-02-07 15:55:59,412 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1 2023-02-07 15:55:59,413 INFO [StoreOpener-db7af127455310d075d9f48aa368dca1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db7af127455310d075d9f48aa368dca1 columnFamilyName cf1 2023-02-07 15:55:59,430 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c-bottom 2023-02-07 15:55:59,433 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/395de32f28b64f6f8705ee57a236e690.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/395de32f28b64f6f8705ee57a236e690-bottom 2023-02-07 15:55:59,436 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/8358a309ce184e3598635c25e33805a2.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/8358a309ce184e3598635c25e33805a2-bottom 2023-02-07 15:55:59,439 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/84fc4227281545bbb7ee3351320e43f9.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/84fc4227281545bbb7ee3351320e43f9-bottom 2023-02-07 15:55:59,442 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/bd5295dc14b04b63b78dcf385fd42064.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/bd5295dc14b04b63b78dcf385fd42064-bottom 2023-02-07 15:55:59,446 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/d3107a7f511843f0af863b0bc529c1bd.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/d3107a7f511843f0af863b0bc529c1bd-bottom 2023-02-07 15:55:59,449 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445-bottom 2023-02-07 15:55:59,449 INFO [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(310): Store=db7af127455310d075d9f48aa368dca1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:59,449 INFO [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region db7af127455310d075d9f48aa368dca1 2023-02-07 15:55:59,450 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2 2023-02-07 15:55:59,450 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2 2023-02-07 15:55:59,450 INFO [StoreOpener-db7af127455310d075d9f48aa368dca1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db7af127455310d075d9f48aa368dca1 columnFamilyName cf2 2023-02-07 15:55:59,462 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/0722de5250584b3abf459a4c95bf348b.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/0722de5250584b3abf459a4c95bf348b-bottom 2023-02-07 15:55:59,465 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/185de16e817e4ae7b80491991f2af9e5.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/185de16e817e4ae7b80491991f2af9e5-bottom 2023-02-07 15:55:59,468 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/54fdf8c1c41a482992c6b9e8fda7b506.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/54fdf8c1c41a482992c6b9e8fda7b506-bottom 2023-02-07 15:55:59,472 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/5b70123e69544a2cb22394b029386535.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/5b70123e69544a2cb22394b029386535-bottom 2023-02-07 15:55:59,475 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/6c57e193e0ed4d5d80dec167921e4dfd.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/6c57e193e0ed4d5d80dec167921e4dfd-bottom 2023-02-07 15:55:59,479 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec-bottom 2023-02-07 15:55:59,484 DEBUG [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/fa6b32b46e7c461ca8db86ad0898239b.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/fa6b32b46e7c461ca8db86ad0898239b-bottom 2023-02-07 15:55:59,484 INFO [StoreOpener-db7af127455310d075d9f48aa368dca1-1] regionserver.HStore(310): Store=db7af127455310d075d9f48aa368dca1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:59,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1 2023-02-07 15:55:59,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1 2023-02-07 15:55:59,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:55:59,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for db7af127455310d075d9f48aa368dca1 2023-02-07 15:55:59,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened db7af127455310d075d9f48aa368dca1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75360796, jitterRate=0.12296336889266968}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:55:59,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for db7af127455310d075d9f48aa368dca1: Running coprocessor pre-open hook at 1675785359409Writing region info on filesystem at 1675785359409Initializing all the Stores at 1675785359410 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785359410Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785359411 (+1 ms)Cleaning up temporary data from old regions at 1675785359486 (+75 ms)Cleaning up detritus from prior splits at 1675785359487 (+1 ms)Running coprocessor post-open hooks at 1675785359488 (+1 ms)Region opened successfully at 1675785359488 2023-02-07 15:55:59,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1., pid=149, masterSystemTime=1675785359405 2023-02-07 15:55:59,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:55:59,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:55:59,490 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:55:59,490 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:55:59,490 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1. 2023-02-07 15:55:59,490 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1. 2023-02-07 15:55:59,490 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1912): db7af127455310d075d9f48aa368dca1/cf1 is initiating minor compaction (all files) 2023-02-07 15:55:59,490 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1912): db7af127455310d075d9f48aa368dca1/cf2 is initiating minor compaction (all files) 2023-02-07 15:55:59,490 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of db7af127455310d075d9f48aa368dca1/cf1 in testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1. 2023-02-07 15:55:59,490 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2258): Starting compaction of db7af127455310d075d9f48aa368dca1/cf2 in testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1. 2023-02-07 15:55:59,490 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/84fc4227281545bbb7ee3351320e43f9.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/84fc4227281545bbb7ee3351320e43f9-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/395de32f28b64f6f8705ee57a236e690.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/395de32f28b64f6f8705ee57a236e690-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/d3107a7f511843f0af863b0bc529c1bd.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/d3107a7f511843f0af863b0bc529c1bd-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/8358a309ce184e3598635c25e33805a2.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/8358a309ce184e3598635c25e33805a2-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/bd5295dc14b04b63b78dcf385fd42064.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/bd5295dc14b04b63b78dcf385fd42064-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445-bottom] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/.tmp, totalSize=34.9 K 2023-02-07 15:55:59,490 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/54fdf8c1c41a482992c6b9e8fda7b506.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/54fdf8c1c41a482992c6b9e8fda7b506-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/0722de5250584b3abf459a4c95bf348b.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/0722de5250584b3abf459a4c95bf348b-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/6c57e193e0ed4d5d80dec167921e4dfd.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/6c57e193e0ed4d5d80dec167921e4dfd-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/fa6b32b46e7c461ca8db86ad0898239b.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/fa6b32b46e7c461ca8db86ad0898239b-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/185de16e817e4ae7b80491991f2af9e5.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/185de16e817e4ae7b80491991f2af9e5-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/5b70123e69544a2cb22394b029386535.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/5b70123e69544a2cb22394b029386535-bottom] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/.tmp, totalSize=34.9 K 2023-02-07 15:55:59,491 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting d7f27d6c055a4e99a53b7a2fd104b9ec.1ef1944f776e9a061c872d33197334a4, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785354970 2023-02-07 15:55:59,491 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 84fc4227281545bbb7ee3351320e43f9.1ef1944f776e9a061c872d33197334a4, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675785354970 2023-02-07 15:55:59,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1. 2023-02-07 15:55:59,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1. 2023-02-07 15:55:59,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4. 2023-02-07 15:55:59,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f10c2c4ee9fd52ac49d41344e75d0dd4, NAME => 'testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.', STARTKEY => '41', ENDKEY => ''} 2023-02-07 15:55:59,491 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 54fdf8c1c41a482992c6b9e8fda7b506.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785355289 2023-02-07 15:55:59,491 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 0abf9da9bc5144e7a8cf28cd08bf6f7c.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675785355289 2023-02-07 15:55:59,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 15:55:59,491 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 0722de5250584b3abf459a4c95bf348b.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785355581 2023-02-07 15:55:59,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:59,491 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 395de32f28b64f6f8705ee57a236e690.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675785355581 2023-02-07 15:55:59,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 15:55:59,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:59,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:59,492 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 6c57e193e0ed4d5d80dec167921e4dfd.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785355833 2023-02-07 15:55:59,492 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting d3107a7f511843f0af863b0bc529c1bd.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675785355833 2023-02-07 15:55:59,492 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 8358a309ce184e3598635c25e33805a2.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785356603 2023-02-07 15:55:59,492 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=db7af127455310d075d9f48aa368dca1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:59,492 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting fa6b32b46e7c461ca8db86ad0898239b.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675785356603 2023-02-07 15:55:59,492 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785359492"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785359492"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785359492"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785359492"}]},"ts":"1675785359492"} 2023-02-07 15:55:59,492 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting bd5295dc14b04b63b78dcf385fd42064.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785358065 2023-02-07 15:55:59,493 INFO [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:59,493 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 185de16e817e4ae7b80491991f2af9e5.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675785358065 2023-02-07 15:55:59,493 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting f4abfa77b0d54ff4a6c2c83ed06ae445.1ef1944f776e9a061c872d33197334a4, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675785358317 2023-02-07 15:55:59,493 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 5b70123e69544a2cb22394b029386535.1ef1944f776e9a061c872d33197334a4, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675785358317 2023-02-07 15:55:59,494 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1 2023-02-07 15:55:59,494 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1 2023-02-07 15:55:59,495 INFO [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f10c2c4ee9fd52ac49d41344e75d0dd4 columnFamilyName cf1 2023-02-07 15:55:59,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-07 15:55:59,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure db7af127455310d075d9f48aa368dca1, server=jenkins-hbase12.apache.org,38629,1675785236797 in 241 msec 2023-02-07 15:55:59,497 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=db7af127455310d075d9f48aa368dca1, ASSIGN in 399 msec 2023-02-07 15:55:59,502 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] throttle.PressureAwareThroughputController(145): db7af127455310d075d9f48aa368dca1#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 61.54 MB/second 2023-02-07 15:55:59,502 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] throttle.PressureAwareThroughputController(145): db7af127455310d075d9f48aa368dca1#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 61.54 MB/second 2023-02-07 15:55:59,505 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c-top 2023-02-07 15:55:59,508 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/395de32f28b64f6f8705ee57a236e690.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/395de32f28b64f6f8705ee57a236e690-top 2023-02-07 15:55:59,511 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/.tmp/cf1/89f5e721f05e4df7a1ab9a3c25608b89 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/89f5e721f05e4df7a1ab9a3c25608b89 2023-02-07 15:55:59,511 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/8358a309ce184e3598635c25e33805a2.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/8358a309ce184e3598635c25e33805a2-top 2023-02-07 15:55:59,511 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/.tmp/cf2/0d88f23288f14b76a140ecb229530b7a as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/0d88f23288f14b76a140ecb229530b7a 2023-02-07 15:55:59,514 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/84fc4227281545bbb7ee3351320e43f9.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/84fc4227281545bbb7ee3351320e43f9-top 2023-02-07 15:55:59,514 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in db7af127455310d075d9f48aa368dca1/cf1 of db7af127455310d075d9f48aa368dca1 into 89f5e721f05e4df7a1ab9a3c25608b89(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:55:59,514 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in db7af127455310d075d9f48aa368dca1/cf2 of db7af127455310d075d9f48aa368dca1 into 0d88f23288f14b76a140ecb229530b7a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:55:59,515 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for db7af127455310d075d9f48aa368dca1: 2023-02-07 15:55:59,515 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for db7af127455310d075d9f48aa368dca1: 2023-02-07 15:55:59,515 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1., storeName=db7af127455310d075d9f48aa368dca1/cf1, priority=9, startTime=1675785359489; duration=0sec 2023-02-07 15:55:59,515 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1., storeName=db7af127455310d075d9f48aa368dca1/cf2, priority=9, startTime=1675785359490; duration=0sec 2023-02-07 15:55:59,515 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:59,515 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:59,516 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/bd5295dc14b04b63b78dcf385fd42064.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/bd5295dc14b04b63b78dcf385fd42064-top 2023-02-07 15:55:59,519 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/d3107a7f511843f0af863b0bc529c1bd.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/d3107a7f511843f0af863b0bc529c1bd-top 2023-02-07 15:55:59,521 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445-top 2023-02-07 15:55:59,521 INFO [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(310): Store=f10c2c4ee9fd52ac49d41344e75d0dd4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:59,521 INFO [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:59,522 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2 2023-02-07 15:55:59,522 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2 2023-02-07 15:55:59,522 INFO [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f10c2c4ee9fd52ac49d41344e75d0dd4 columnFamilyName cf2 2023-02-07 15:55:59,532 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/0722de5250584b3abf459a4c95bf348b.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/0722de5250584b3abf459a4c95bf348b-top 2023-02-07 15:55:59,535 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/185de16e817e4ae7b80491991f2af9e5.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/185de16e817e4ae7b80491991f2af9e5-top 2023-02-07 15:55:59,537 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/54fdf8c1c41a482992c6b9e8fda7b506.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/54fdf8c1c41a482992c6b9e8fda7b506-top 2023-02-07 15:55:59,540 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/5b70123e69544a2cb22394b029386535.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/5b70123e69544a2cb22394b029386535-top 2023-02-07 15:55:59,542 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/6c57e193e0ed4d5d80dec167921e4dfd.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/6c57e193e0ed4d5d80dec167921e4dfd-top 2023-02-07 15:55:59,544 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec-top 2023-02-07 15:55:59,547 DEBUG [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/fa6b32b46e7c461ca8db86ad0898239b.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/fa6b32b46e7c461ca8db86ad0898239b-top 2023-02-07 15:55:59,547 INFO [StoreOpener-f10c2c4ee9fd52ac49d41344e75d0dd4-1] regionserver.HStore(310): Store=f10c2c4ee9fd52ac49d41344e75d0dd4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 15:55:59,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:59,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:59,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 15:55:59,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:59,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened f10c2c4ee9fd52ac49d41344e75d0dd4; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66124157, jitterRate=-0.014673277735710144}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 15:55:59,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for f10c2c4ee9fd52ac49d41344e75d0dd4: Running coprocessor pre-open hook at 1675785359492Writing region info on filesystem at 1675785359492Initializing all the Stores at 1675785359492Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785359492Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675785359492Cleaning up temporary data from old regions at 1675785359549 (+57 ms)Cleaning up detritus from prior splits at 1675785359549Running coprocessor post-open hooks at 1675785359551 (+2 ms)Region opened successfully at 1675785359551 2023-02-07 15:55:59,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4., pid=150, masterSystemTime=1675785359405 2023-02-07 15:55:59,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:55:59,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 15:55:59,552 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:55:59,552 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 15:55:59,552 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4. 2023-02-07 15:55:59,552 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4. 2023-02-07 15:55:59,552 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1912): f10c2c4ee9fd52ac49d41344e75d0dd4/cf1 is initiating minor compaction (all files) 2023-02-07 15:55:59,552 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1912): f10c2c4ee9fd52ac49d41344e75d0dd4/cf2 is initiating minor compaction (all files) 2023-02-07 15:55:59,552 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f10c2c4ee9fd52ac49d41344e75d0dd4/cf1 in testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4. 2023-02-07 15:55:59,552 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f10c2c4ee9fd52ac49d41344e75d0dd4/cf2 in testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4. 2023-02-07 15:55:59,553 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/54fdf8c1c41a482992c6b9e8fda7b506.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/54fdf8c1c41a482992c6b9e8fda7b506-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/0722de5250584b3abf459a4c95bf348b.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/0722de5250584b3abf459a4c95bf348b-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/6c57e193e0ed4d5d80dec167921e4dfd.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/6c57e193e0ed4d5d80dec167921e4dfd-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/fa6b32b46e7c461ca8db86ad0898239b.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/fa6b32b46e7c461ca8db86ad0898239b-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/185de16e817e4ae7b80491991f2af9e5.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/185de16e817e4ae7b80491991f2af9e5-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/5b70123e69544a2cb22394b029386535.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/5b70123e69544a2cb22394b029386535-top] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/.tmp, totalSize=34.9 K 2023-02-07 15:55:59,553 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/84fc4227281545bbb7ee3351320e43f9.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/84fc4227281545bbb7ee3351320e43f9-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/395de32f28b64f6f8705ee57a236e690.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/395de32f28b64f6f8705ee57a236e690-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/d3107a7f511843f0af863b0bc529c1bd.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/d3107a7f511843f0af863b0bc529c1bd-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/8358a309ce184e3598635c25e33805a2.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/8358a309ce184e3598635c25e33805a2-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/bd5295dc14b04b63b78dcf385fd42064.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/bd5295dc14b04b63b78dcf385fd42064-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445-top] into tmpdir=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/.tmp, totalSize=34.9 K 2023-02-07 15:55:59,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4. 2023-02-07 15:55:59,554 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting d7f27d6c055a4e99a53b7a2fd104b9ec.1ef1944f776e9a061c872d33197334a4, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675785354970 2023-02-07 15:55:59,554 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 84fc4227281545bbb7ee3351320e43f9.1ef1944f776e9a061c872d33197334a4, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675785354970 2023-02-07 15:55:59,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4. 2023-02-07 15:55:59,554 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=f10c2c4ee9fd52ac49d41344e75d0dd4, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:59,554 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 54fdf8c1c41a482992c6b9e8fda7b506.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675785355289 2023-02-07 15:55:59,554 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785359554"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675785359554"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675785359554"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675785359554"}]},"ts":"1675785359554"} 2023-02-07 15:55:59,554 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 0abf9da9bc5144e7a8cf28cd08bf6f7c.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675785355289 2023-02-07 15:55:59,554 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 0722de5250584b3abf459a4c95bf348b.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675785355581 2023-02-07 15:55:59,554 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 395de32f28b64f6f8705ee57a236e690.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675785355581 2023-02-07 15:55:59,555 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting d3107a7f511843f0af863b0bc529c1bd.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675785355833 2023-02-07 15:55:59,555 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 6c57e193e0ed4d5d80dec167921e4dfd.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675785355833 2023-02-07 15:55:59,555 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting 8358a309ce184e3598635c25e33805a2.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675785356603 2023-02-07 15:55:59,555 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting fa6b32b46e7c461ca8db86ad0898239b.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675785356603 2023-02-07 15:55:59,555 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting bd5295dc14b04b63b78dcf385fd42064.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675785358065 2023-02-07 15:55:59,556 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 185de16e817e4ae7b80491991f2af9e5.1ef1944f776e9a061c872d33197334a4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675785358065 2023-02-07 15:55:59,556 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] compactions.Compactor(207): Compacting f4abfa77b0d54ff4a6c2c83ed06ae445.1ef1944f776e9a061c872d33197334a4, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675785358317 2023-02-07 15:55:59,556 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] compactions.Compactor(207): Compacting 5b70123e69544a2cb22394b029386535.1ef1944f776e9a061c872d33197334a4, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675785358317 2023-02-07 15:55:59,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-07 15:55:59,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure f10c2c4ee9fd52ac49d41344e75d0dd4, server=jenkins-hbase12.apache.org,38629,1675785236797 in 297 msec 2023-02-07 15:55:59,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-07 15:55:59,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=f10c2c4ee9fd52ac49d41344e75d0dd4, ASSIGN in 460 msec 2023-02-07 15:55:59,560 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=1ef1944f776e9a061c872d33197334a4, daughterA=db7af127455310d075d9f48aa368dca1, daughterB=f10c2c4ee9fd52ac49d41344e75d0dd4 in 1.2360 sec 2023-02-07 15:55:59,564 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] throttle.PressureAwareThroughputController(145): f10c2c4ee9fd52ac49d41344e75d0dd4#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 61.54 MB/second 2023-02-07 15:55:59,564 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] throttle.PressureAwareThroughputController(145): f10c2c4ee9fd52ac49d41344e75d0dd4#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 61.54 MB/second 2023-02-07 15:55:59,574 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/.tmp/cf1/202fce98a7934900b0fe8d6097f924a1 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/202fce98a7934900b0fe8d6097f924a1 2023-02-07 15:55:59,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38629] ipc.CallRunner(144): callId: 1124 service: ClientService methodName: Scan size: 143 connection: 136.243.104.168:51088 deadline: 1675785419575, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4. is not online on jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:59,575 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/.tmp/cf2/bce4159f57cd4f1da96d63f6deebd37f as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/bce4159f57cd4f1da96d63f6deebd37f 2023-02-07 15:55:59,579 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f10c2c4ee9fd52ac49d41344e75d0dd4/cf1 of f10c2c4ee9fd52ac49d41344e75d0dd4 into 202fce98a7934900b0fe8d6097f924a1(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:55:59,579 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f10c2c4ee9fd52ac49d41344e75d0dd4/cf2 of f10c2c4ee9fd52ac49d41344e75d0dd4 into bce4159f57cd4f1da96d63f6deebd37f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 15:55:59,579 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f10c2c4ee9fd52ac49d41344e75d0dd4: 2023-02-07 15:55:59,579 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f10c2c4ee9fd52ac49d41344e75d0dd4: 2023-02-07 15:55:59,579 INFO [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4., storeName=f10c2c4ee9fd52ac49d41344e75d0dd4/cf1, priority=9, startTime=1675785359551; duration=0sec 2023-02-07 15:55:59,579 INFO [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4., storeName=f10c2c4ee9fd52ac49d41344e75d0dd4/cf2, priority=9, startTime=1675785359552; duration=0sec 2023-02-07 15:55:59,579 DEBUG [RS:0;jenkins-hbase12:38629-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:59,579 DEBUG [RS:0;jenkins-hbase12:38629-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 15:55:59,690 DEBUG [Listener at localhost.localdomain/41965] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 15:55:59,700 INFO [Listener at localhost.localdomain/41965] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-07 15:55:59,702 DEBUG [Listener at localhost.localdomain/41965] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 15:55:59,710 WARN [Listener at localhost.localdomain/41965] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 15:55:59,710 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:59,710 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:59,712 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:59,712 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:59,712 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-07 15:55:59,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegion 2023-02-07 15:55:59,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-07 15:55:59,715 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:59,715 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:59,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-07 15:55:59,715 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785359715"}]},"ts":"1675785359715"} 2023-02-07 15:55:59,716 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-07 15:55:59,762 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-07 15:55:59,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=db7af127455310d075d9f48aa368dca1, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=f10c2c4ee9fd52ac49d41344e75d0dd4, UNASSIGN}] 2023-02-07 15:55:59,764 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=f10c2c4ee9fd52ac49d41344e75d0dd4, UNASSIGN 2023-02-07 15:55:59,764 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=f10c2c4ee9fd52ac49d41344e75d0dd4, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:59,764 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785359764"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785359764"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785359764"}]},"ts":"1675785359764"} 2023-02-07 15:55:59,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure f10c2c4ee9fd52ac49d41344e75d0dd4, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:59,767 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=db7af127455310d075d9f48aa368dca1, UNASSIGN 2023-02-07 15:55:59,767 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=db7af127455310d075d9f48aa368dca1, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:55:59,767 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785359767"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675785359767"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675785359767"}]},"ts":"1675785359767"} 2023-02-07 15:55:59,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure db7af127455310d075d9f48aa368dca1, server=jenkins-hbase12.apache.org,38629,1675785236797}] 2023-02-07 15:55:59,815 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:55:59,816 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:55:59,971 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:59,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing f10c2c4ee9fd52ac49d41344e75d0dd4, disabling compactions & flushes 2023-02-07 15:55:59,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-07 15:55:59,971 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4. 2023-02-07 15:55:59,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4. 2023-02-07 15:55:59,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4. after waiting 0 ms 2023-02-07 15:55:59,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4. 2023-02-07 15:55:59,972 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/84fc4227281545bbb7ee3351320e43f9.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/84fc4227281545bbb7ee3351320e43f9-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/395de32f28b64f6f8705ee57a236e690.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/395de32f28b64f6f8705ee57a236e690-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/d3107a7f511843f0af863b0bc529c1bd.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/d3107a7f511843f0af863b0bc529c1bd-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/8358a309ce184e3598635c25e33805a2.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/8358a309ce184e3598635c25e33805a2-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/bd5295dc14b04b63b78dcf385fd42064.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/bd5295dc14b04b63b78dcf385fd42064-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445-top] to archive 2023-02-07 15:55:59,972 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:55:59,974 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/84fc4227281545bbb7ee3351320e43f9.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/84fc4227281545bbb7ee3351320e43f9.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,975 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,976 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/395de32f28b64f6f8705ee57a236e690.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/395de32f28b64f6f8705ee57a236e690.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,977 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/d3107a7f511843f0af863b0bc529c1bd.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/d3107a7f511843f0af863b0bc529c1bd.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,978 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/8358a309ce184e3598635c25e33805a2.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/8358a309ce184e3598635c25e33805a2.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,979 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/bd5295dc14b04b63b78dcf385fd42064.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/bd5295dc14b04b63b78dcf385fd42064.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,980 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,981 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/54fdf8c1c41a482992c6b9e8fda7b506.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/54fdf8c1c41a482992c6b9e8fda7b506-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/0722de5250584b3abf459a4c95bf348b.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/0722de5250584b3abf459a4c95bf348b-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/6c57e193e0ed4d5d80dec167921e4dfd.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/6c57e193e0ed4d5d80dec167921e4dfd-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/fa6b32b46e7c461ca8db86ad0898239b.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/fa6b32b46e7c461ca8db86ad0898239b-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/185de16e817e4ae7b80491991f2af9e5.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/185de16e817e4ae7b80491991f2af9e5-top, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/5b70123e69544a2cb22394b029386535.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/5b70123e69544a2cb22394b029386535-top] to archive 2023-02-07 15:55:59,981 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:55:59,982 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,983 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/54fdf8c1c41a482992c6b9e8fda7b506.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/54fdf8c1c41a482992c6b9e8fda7b506.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,983 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/0722de5250584b3abf459a4c95bf348b.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/0722de5250584b3abf459a4c95bf348b.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,984 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/6c57e193e0ed4d5d80dec167921e4dfd.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/6c57e193e0ed4d5d80dec167921e4dfd.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,985 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/fa6b32b46e7c461ca8db86ad0898239b.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/fa6b32b46e7c461ca8db86ad0898239b.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,985 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/185de16e817e4ae7b80491991f2af9e5.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/185de16e817e4ae7b80491991f2af9e5.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,986 DEBUG [StoreCloser-testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/5b70123e69544a2cb22394b029386535.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/5b70123e69544a2cb22394b029386535.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,989 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 15:55:59,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:55:59,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4. 2023-02-07 15:55:59,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for f10c2c4ee9fd52ac49d41344e75d0dd4: Waiting for close lock at 1675785359971Running coprocessor pre-close hooks at 1675785359971Disabling compacts and flushes for region at 1675785359971Disabling writes for close at 1675785359971Writing region close event to WAL at 1675785359987 (+16 ms)Running coprocessor post-close hooks at 1675785359990 (+3 ms)Closed at 1675785359990 2023-02-07 15:55:59,992 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:55:59,992 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close db7af127455310d075d9f48aa368dca1 2023-02-07 15:55:59,992 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing db7af127455310d075d9f48aa368dca1, disabling compactions & flushes 2023-02-07 15:55:59,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1. 2023-02-07 15:55:59,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1. 2023-02-07 15:55:59,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1. after waiting 0 ms 2023-02-07 15:55:59,993 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=f10c2c4ee9fd52ac49d41344e75d0dd4, regionState=CLOSED 2023-02-07 15:55:59,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1. 2023-02-07 15:55:59,993 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785359993"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785359993"}]},"ts":"1675785359993"} 2023-02-07 15:55:59,994 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/84fc4227281545bbb7ee3351320e43f9.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/84fc4227281545bbb7ee3351320e43f9-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/395de32f28b64f6f8705ee57a236e690.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/395de32f28b64f6f8705ee57a236e690-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/d3107a7f511843f0af863b0bc529c1bd.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/d3107a7f511843f0af863b0bc529c1bd-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/8358a309ce184e3598635c25e33805a2.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/8358a309ce184e3598635c25e33805a2-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/bd5295dc14b04b63b78dcf385fd42064.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/bd5295dc14b04b63b78dcf385fd42064-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445-bottom] to archive 2023-02-07 15:55:59,995 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:55:59,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-07 15:55:59,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure f10c2c4ee9fd52ac49d41344e75d0dd4, server=jenkins-hbase12.apache.org,38629,1675785236797 in 229 msec 2023-02-07 15:55:59,996 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/84fc4227281545bbb7ee3351320e43f9.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/84fc4227281545bbb7ee3351320e43f9.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=f10c2c4ee9fd52ac49d41344e75d0dd4, UNASSIGN in 233 msec 2023-02-07 15:55:59,997 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:55:59,999 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/395de32f28b64f6f8705ee57a236e690.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/395de32f28b64f6f8705ee57a236e690.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,000 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/d3107a7f511843f0af863b0bc529c1bd.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/d3107a7f511843f0af863b0bc529c1bd.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,001 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/8358a309ce184e3598635c25e33805a2.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/8358a309ce184e3598635c25e33805a2.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,002 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/bd5295dc14b04b63b78dcf385fd42064.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/bd5295dc14b04b63b78dcf385fd42064.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,003 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,005 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/54fdf8c1c41a482992c6b9e8fda7b506.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/54fdf8c1c41a482992c6b9e8fda7b506-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/0722de5250584b3abf459a4c95bf348b.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/0722de5250584b3abf459a4c95bf348b-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/6c57e193e0ed4d5d80dec167921e4dfd.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/6c57e193e0ed4d5d80dec167921e4dfd-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/fa6b32b46e7c461ca8db86ad0898239b.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/fa6b32b46e7c461ca8db86ad0898239b-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/185de16e817e4ae7b80491991f2af9e5.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/185de16e817e4ae7b80491991f2af9e5-bottom, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/5b70123e69544a2cb22394b029386535.1ef1944f776e9a061c872d33197334a4->hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/5b70123e69544a2cb22394b029386535-bottom] to archive 2023-02-07 15:56:00,005 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 15:56:00,007 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,008 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/54fdf8c1c41a482992c6b9e8fda7b506.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/54fdf8c1c41a482992c6b9e8fda7b506.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,009 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/0722de5250584b3abf459a4c95bf348b.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/0722de5250584b3abf459a4c95bf348b.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,011 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/6c57e193e0ed4d5d80dec167921e4dfd.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/6c57e193e0ed4d5d80dec167921e4dfd.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,012 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/fa6b32b46e7c461ca8db86ad0898239b.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/fa6b32b46e7c461ca8db86ad0898239b.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,013 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/185de16e817e4ae7b80491991f2af9e5.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/185de16e817e4ae7b80491991f2af9e5.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,015 DEBUG [StoreCloser-testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/5b70123e69544a2cb22394b029386535.1ef1944f776e9a061c872d33197334a4 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/5b70123e69544a2cb22394b029386535.1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 15:56:00,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:56:00,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1. 2023-02-07 15:56:00,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for db7af127455310d075d9f48aa368dca1: Waiting for close lock at 1675785359992Running coprocessor pre-close hooks at 1675785359992Disabling compacts and flushes for region at 1675785359992Disabling writes for close at 1675785359993 (+1 ms)Writing region close event to WAL at 1675785360016 (+23 ms)Running coprocessor post-close hooks at 1675785360019 (+3 ms)Closed at 1675785360019 2023-02-07 15:56:00,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed db7af127455310d075d9f48aa368dca1 2023-02-07 15:56:00,021 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=db7af127455310d075d9f48aa368dca1, regionState=CLOSED 2023-02-07 15:56:00,021 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675785360021"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675785360021"}]},"ts":"1675785360021"} 2023-02-07 15:56:00,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-07 15:56:00,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure db7af127455310d075d9f48aa368dca1, server=jenkins-hbase12.apache.org,38629,1675785236797 in 254 msec 2023-02-07 15:56:00,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-07 15:56:00,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=db7af127455310d075d9f48aa368dca1, UNASSIGN in 261 msec 2023-02-07 15:56:00,027 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675785360027"}]},"ts":"1675785360027"} 2023-02-07 15:56:00,028 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-07 15:56:00,036 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-07 15:56:00,038 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 324 msec 2023-02-07 15:56:00,171 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:56:00,171 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:56:00,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-07 15:56:00,174 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-07 15:56:00,174 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:56:00,174 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:56:00,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegion 2023-02-07 15:56:00,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-07 15:56:00,180 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-07 15:56:00,180 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:56:00,180 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:56:00,180 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-07 15:56:00,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-07 15:56:00,186 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,186 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:56:00,186 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1 2023-02-07 15:56:00,189 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/recovered.edits] 2023-02-07 15:56:00,189 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/recovered.edits] 2023-02-07 15:56:00,189 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2, FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/recovered.edits] 2023-02-07 15:56:00,193 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/202fce98a7934900b0fe8d6097f924a1 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf1/202fce98a7934900b0fe8d6097f924a1 2023-02-07 15:56:00,193 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/89f5e721f05e4df7a1ab9a3c25608b89 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf1/89f5e721f05e4df7a1ab9a3c25608b89 2023-02-07 15:56:00,194 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/0abf9da9bc5144e7a8cf28cd08bf6f7c 2023-02-07 15:56:00,194 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/395de32f28b64f6f8705ee57a236e690 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/395de32f28b64f6f8705ee57a236e690 2023-02-07 15:56:00,195 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/bce4159f57cd4f1da96d63f6deebd37f to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/cf2/bce4159f57cd4f1da96d63f6deebd37f 2023-02-07 15:56:00,195 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/0d88f23288f14b76a140ecb229530b7a to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/cf2/0d88f23288f14b76a140ecb229530b7a 2023-02-07 15:56:00,195 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/8358a309ce184e3598635c25e33805a2 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/8358a309ce184e3598635c25e33805a2 2023-02-07 15:56:00,196 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/84fc4227281545bbb7ee3351320e43f9 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/84fc4227281545bbb7ee3351320e43f9 2023-02-07 15:56:00,197 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/recovered.edits/90.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4/recovered.edits/90.seqid 2023-02-07 15:56:00,197 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/recovered.edits/90.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1/recovered.edits/90.seqid 2023-02-07 15:56:00,198 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/f10c2c4ee9fd52ac49d41344e75d0dd4 2023-02-07 15:56:00,198 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/db7af127455310d075d9f48aa368dca1 2023-02-07 15:56:00,198 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/bd5295dc14b04b63b78dcf385fd42064 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/bd5295dc14b04b63b78dcf385fd42064 2023-02-07 15:56:00,199 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/d3107a7f511843f0af863b0bc529c1bd to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/d3107a7f511843f0af863b0bc529c1bd 2023-02-07 15:56:00,199 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf1/f4abfa77b0d54ff4a6c2c83ed06ae445 2023-02-07 15:56:00,201 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/0722de5250584b3abf459a4c95bf348b to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/0722de5250584b3abf459a4c95bf348b 2023-02-07 15:56:00,202 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/185de16e817e4ae7b80491991f2af9e5 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/185de16e817e4ae7b80491991f2af9e5 2023-02-07 15:56:00,202 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/54fdf8c1c41a482992c6b9e8fda7b506 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/54fdf8c1c41a482992c6b9e8fda7b506 2023-02-07 15:56:00,203 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/5b70123e69544a2cb22394b029386535 to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/5b70123e69544a2cb22394b029386535 2023-02-07 15:56:00,204 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/6c57e193e0ed4d5d80dec167921e4dfd to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/6c57e193e0ed4d5d80dec167921e4dfd 2023-02-07 15:56:00,204 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/d7f27d6c055a4e99a53b7a2fd104b9ec 2023-02-07 15:56:00,205 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/fa6b32b46e7c461ca8db86ad0898239b to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/cf2/fa6b32b46e7c461ca8db86ad0898239b 2023-02-07 15:56:00,208 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/recovered.edits/85.seqid to hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/archive/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4/recovered.edits/85.seqid 2023-02-07 15:56:00,208 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/.tmp/data/default/testSplitTableRegion/1ef1944f776e9a061c872d33197334a4 2023-02-07 15:56:00,208 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-07 15:56:00,210 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-07 15:56:00,212 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-07 15:56:00,214 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-07 15:56:00,214 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-07 15:56:00,214 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-07 15:56:00,215 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785360215"}]},"ts":"9223372036854775807"} 2023-02-07 15:56:00,215 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785360215"}]},"ts":"9223372036854775807"} 2023-02-07 15:56:00,215 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675785360215"}]},"ts":"9223372036854775807"} 2023-02-07 15:56:00,216 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 15:56:00,217 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1ef1944f776e9a061c872d33197334a4, NAME => 'testSplitTableRegion,,1675785353963.1ef1944f776e9a061c872d33197334a4.', STARTKEY => '', ENDKEY => ''}, {ENCODED => db7af127455310d075d9f48aa368dca1, NAME => 'testSplitTableRegion,,1675785358323.db7af127455310d075d9f48aa368dca1.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => f10c2c4ee9fd52ac49d41344e75d0dd4, NAME => 'testSplitTableRegion,41,1675785358323.f10c2c4ee9fd52ac49d41344e75d0dd4.', STARTKEY => '41', ENDKEY => ''}] 2023-02-07 15:56:00,217 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-07 15:56:00,217 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675785360217"}]},"ts":"9223372036854775807"} 2023-02-07 15:56:00,218 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-07 15:56:00,226 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-07 15:56:00,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 50 msec 2023-02-07 15:56:00,282 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 15:56:00,282 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 15:56:00,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37073] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-07 15:56:00,283 INFO [Listener at localhost.localdomain/41965] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-07 15:56:00,299 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 15:56:00,302 INFO [Listener at localhost.localdomain/41965] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=378 (was 374) - Thread LEAK? -, OpenFileDescriptor=844 (was 847), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=183 (was 173) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=2794 (was 2908) 2023-02-07 15:56:00,303 INFO [Listener at localhost.localdomain/41965] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-07 15:56:00,303 INFO [Listener at localhost.localdomain/41965] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-07 15:56:00,304 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7900af31 to 127.0.0.1:51095 2023-02-07 15:56:00,304 DEBUG [Listener at localhost.localdomain/41965] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 15:56:00,305 DEBUG [Listener at localhost.localdomain/41965] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-07 15:56:00,305 DEBUG [Listener at localhost.localdomain/41965] util.JVMClusterUtil(257): Found active master hash=2138838705, stopped=false 2023-02-07 15:56:00,305 INFO [Listener at localhost.localdomain/41965] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase12.apache.org,37073,1675785235583 2023-02-07 15:56:00,317 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-07 15:56:00,317 INFO [Listener at localhost.localdomain/41965] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 15:56:00,317 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-07 15:56:00,317 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 15:56:00,317 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-07 15:56:00,317 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-07 15:56:00,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 15:56:00,318 DEBUG [Listener at localhost.localdomain/41965] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x79800e45 to 127.0.0.1:51095 2023-02-07 15:56:00,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 15:56:00,318 DEBUG [Listener at localhost.localdomain/41965] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 15:56:00,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 15:56:00,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 15:56:00,318 INFO [Listener at localhost.localdomain/41965] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,38629,1675785236797' ***** 2023-02-07 15:56:00,318 INFO [Listener at localhost.localdomain/41965] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-07 15:56:00,318 INFO [Listener at localhost.localdomain/41965] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,35657,1675785236854' ***** 2023-02-07 15:56:00,318 INFO [Listener at localhost.localdomain/41965] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-07 15:56:00,318 INFO [Listener at localhost.localdomain/41965] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,37399,1675785236898' ***** 2023-02-07 15:56:00,318 INFO [Listener at localhost.localdomain/41965] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-07 15:56:00,318 INFO [RS:1;jenkins-hbase12:35657] regionserver.HeapMemoryManager(220): Stopping 2023-02-07 15:56:00,319 INFO [RS:2;jenkins-hbase12:37399] regionserver.HeapMemoryManager(220): Stopping 2023-02-07 15:56:00,318 INFO [RS:0;jenkins-hbase12:38629] regionserver.HeapMemoryManager(220): Stopping 2023-02-07 15:56:00,319 INFO [RS:0;jenkins-hbase12:38629] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-07 15:56:00,319 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-07 15:56:00,319 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-07 15:56:00,319 INFO [RS:2;jenkins-hbase12:37399] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-07 15:56:00,319 INFO [RS:1;jenkins-hbase12:35657] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-07 15:56:00,319 INFO [RS:0;jenkins-hbase12:38629] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-07 15:56:00,319 INFO [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(3304): Received CLOSE for 2b3e2412ef7ea40bb26319d571a3f2e6 2023-02-07 15:56:00,319 INFO [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:56:00,320 DEBUG [RS:0;jenkins-hbase12:38629] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x38ca98b9 to 127.0.0.1:51095 2023-02-07 15:56:00,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 2b3e2412ef7ea40bb26319d571a3f2e6, disabling compactions & flushes 2023-02-07 15:56:00,319 INFO [RS:2;jenkins-hbase12:37399] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-07 15:56:00,319 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-07 15:56:00,320 INFO [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:56:00,320 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. 2023-02-07 15:56:00,320 DEBUG [RS:0;jenkins-hbase12:38629] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 15:56:00,319 INFO [RS:1;jenkins-hbase12:35657] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-07 15:56:00,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. 2023-02-07 15:56:00,320 DEBUG [RS:2;jenkins-hbase12:37399] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x738961ee to 127.0.0.1:51095 2023-02-07 15:56:00,321 DEBUG [RS:2;jenkins-hbase12:37399] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 15:56:00,321 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. after waiting 0 ms 2023-02-07 15:56:00,322 INFO [RS:2;jenkins-hbase12:37399] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-07 15:56:00,321 INFO [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-07 15:56:00,321 INFO [RS:1;jenkins-hbase12:35657] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:56:00,322 DEBUG [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(1479): Online Regions={2b3e2412ef7ea40bb26319d571a3f2e6=hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6.} 2023-02-07 15:56:00,322 INFO [RS:2;jenkins-hbase12:37399] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-07 15:56:00,322 INFO [RS:2;jenkins-hbase12:37399] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-07 15:56:00,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. 2023-02-07 15:56:00,322 INFO [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-07 15:56:00,322 DEBUG [RS:1;jenkins-hbase12:35657] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4474c8fd to 127.0.0.1:51095 2023-02-07 15:56:00,322 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 2b3e2412ef7ea40bb26319d571a3f2e6 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-07 15:56:00,322 DEBUG [RS:1;jenkins-hbase12:35657] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 15:56:00,323 INFO [RS:1;jenkins-hbase12:35657] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,35657,1675785236854; all regions closed. 2023-02-07 15:56:00,323 INFO [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-07 15:56:00,323 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-07 15:56:00,323 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-07 15:56:00,323 DEBUG [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-07 15:56:00,323 DEBUG [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(1505): Waiting on 2b3e2412ef7ea40bb26319d571a3f2e6 2023-02-07 15:56:00,323 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-07 15:56:00,323 DEBUG [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-07 15:56:00,324 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-07 15:56:00,324 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-07 15:56:00,324 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.49 KB heapSize=125.82 KB 2023-02-07 15:56:00,348 DEBUG [RS:1;jenkins-hbase12:35657] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/oldWALs 2023-02-07 15:56:00,348 INFO [RS:1;jenkins-hbase12:35657] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C35657%2C1675785236854:(num 1675785238564) 2023-02-07 15:56:00,348 DEBUG [RS:1;jenkins-hbase12:35657] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 15:56:00,348 INFO [RS:1;jenkins-hbase12:35657] regionserver.LeaseManager(133): Closed leases 2023-02-07 15:56:00,349 INFO [RS:1;jenkins-hbase12:35657] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase12:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-07 15:56:00,349 INFO [RS:1;jenkins-hbase12:35657] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-07 15:56:00,349 INFO [regionserver/jenkins-hbase12:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-07 15:56:00,349 INFO [RS:1;jenkins-hbase12:35657] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-07 15:56:00,349 INFO [RS:1;jenkins-hbase12:35657] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-07 15:56:00,350 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6/.tmp/info/394500092b784b9995096e6eb546b61c 2023-02-07 15:56:00,351 INFO [RS:1;jenkins-hbase12:35657] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:35657 2023-02-07 15:56:00,353 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.37 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/.tmp/info/6eaab6b0b9f04f0caf1ed2d0ae98d01a 2023-02-07 15:56:00,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6/.tmp/info/394500092b784b9995096e6eb546b61c as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6/info/394500092b784b9995096e6eb546b61c 2023-02-07 15:56:00,359 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6eaab6b0b9f04f0caf1ed2d0ae98d01a 2023-02-07 15:56:00,362 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:56:00,362 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-07 15:56:00,362 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6/info/394500092b784b9995096e6eb546b61c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 15:56:00,362 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-07 15:56:00,362 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:56:00,363 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-07 15:56:00,363 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,35657,1675785236854 2023-02-07 15:56:00,363 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-07 15:56:00,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 2b3e2412ef7ea40bb26319d571a3f2e6 in 41ms, sequenceid=6, compaction requested=false 2023-02-07 15:56:00,364 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase12.apache.org,35657,1675785236854] 2023-02-07 15:56:00,364 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase12.apache.org,35657,1675785236854; numProcessing=1 2023-02-07 15:56:00,370 INFO [regionserver/jenkins-hbase12:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-07 15:56:00,370 INFO [regionserver/jenkins-hbase12:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-07 15:56:00,370 INFO [regionserver/jenkins-hbase12:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-07 15:56:00,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/namespace/2b3e2412ef7ea40bb26319d571a3f2e6/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-07 15:56:00,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:56:00,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. 2023-02-07 15:56:00,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 2b3e2412ef7ea40bb26319d571a3f2e6: Waiting for close lock at 1675785360320Running coprocessor pre-close hooks at 1675785360320Disabling compacts and flushes for region at 1675785360320Disabling writes for close at 1675785360322 (+2 ms)Obtaining lock to block concurrent updates at 1675785360322Preparing flush snapshotting stores in 2b3e2412ef7ea40bb26319d571a3f2e6 at 1675785360322Finished memstore snapshotting hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1675785360323 (+1 ms)Flushing stores of hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. at 1675785360324 (+1 ms)Flushing 2b3e2412ef7ea40bb26319d571a3f2e6/info: creating writer at 1675785360324Flushing 2b3e2412ef7ea40bb26319d571a3f2e6/info: appending metadata at 1675785360327 (+3 ms)Flushing 2b3e2412ef7ea40bb26319d571a3f2e6/info: closing flushed file at 1675785360327Flushing 2b3e2412ef7ea40bb26319d571a3f2e6/info: reopening flushed file at 1675785360359 (+32 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 2b3e2412ef7ea40bb26319d571a3f2e6 in 41ms, sequenceid=6, compaction requested=false at 1675785360363 (+4 ms)Writing region close event to WAL at 1675785360371 (+8 ms)Running coprocessor post-close hooks at 1675785360377 (+6 ms)Closed at 1675785360378 (+1 ms) 2023-02-07 15:56:00,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675785239055.2b3e2412ef7ea40bb26319d571a3f2e6. 2023-02-07 15:56:00,380 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/.tmp/rep_barrier/b0b94ad7688a4b7bafd8bd1ddc8caabc 2023-02-07 15:56:00,383 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b0b94ad7688a4b7bafd8bd1ddc8caabc 2023-02-07 15:56:00,383 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase12.apache.org,35657,1675785236854 already deleted, retry=false 2023-02-07 15:56:00,383 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase12.apache.org,35657,1675785236854 expired; onlineServers=2 2023-02-07 15:56:00,399 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/.tmp/table/1dd73824f61a449bbbf10a5229d597d7 2023-02-07 15:56:00,402 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1dd73824f61a449bbbf10a5229d597d7 2023-02-07 15:56:00,402 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/.tmp/info/6eaab6b0b9f04f0caf1ed2d0ae98d01a as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/info/6eaab6b0b9f04f0caf1ed2d0ae98d01a 2023-02-07 15:56:00,406 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6eaab6b0b9f04f0caf1ed2d0ae98d01a 2023-02-07 15:56:00,406 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/info/6eaab6b0b9f04f0caf1ed2d0ae98d01a, entries=59, sequenceid=210, filesize=11.1 K 2023-02-07 15:56:00,407 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/.tmp/rep_barrier/b0b94ad7688a4b7bafd8bd1ddc8caabc as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/rep_barrier/b0b94ad7688a4b7bafd8bd1ddc8caabc 2023-02-07 15:56:00,411 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b0b94ad7688a4b7bafd8bd1ddc8caabc 2023-02-07 15:56:00,411 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/rep_barrier/b0b94ad7688a4b7bafd8bd1ddc8caabc, entries=24, sequenceid=210, filesize=7.5 K 2023-02-07 15:56:00,412 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/.tmp/table/1dd73824f61a449bbbf10a5229d597d7 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/table/1dd73824f61a449bbbf10a5229d597d7 2023-02-07 15:56:00,412 INFO [regionserver/jenkins-hbase12:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-07 15:56:00,412 INFO [regionserver/jenkins-hbase12:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-07 15:56:00,414 INFO [regionserver/jenkins-hbase12:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-07 15:56:00,414 INFO [regionserver/jenkins-hbase12:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-07 15:56:00,415 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1dd73824f61a449bbbf10a5229d597d7 2023-02-07 15:56:00,415 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/table/1dd73824f61a449bbbf10a5229d597d7, entries=36, sequenceid=210, filesize=8.0 K 2023-02-07 15:56:00,415 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 91ms, sequenceid=210, compaction requested=false 2023-02-07 15:56:00,419 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-07 15:56:00,419 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 15:56:00,420 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-07 15:56:00,420 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-07 15:56:00,420 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675785360323Running coprocessor pre-close hooks at 1675785360323Disabling compacts and flushes for region at 1675785360323Disabling writes for close at 1675785360324 (+1 ms)Obtaining lock to block concurrent updates at 1675785360324Preparing flush snapshotting stores in 1588230740 at 1675785360324Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81396, getHeapSize=128792, getOffHeapSize=0, getCellsCount=617 at 1675785360324Flushing stores of hbase:meta,,1.1588230740 at 1675785360325 (+1 ms)Flushing 1588230740/info: creating writer at 1675785360325Flushing 1588230740/info: appending metadata at 1675785360332 (+7 ms)Flushing 1588230740/info: closing flushed file at 1675785360332Flushing 1588230740/rep_barrier: creating writer at 1675785360359 (+27 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675785360361 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675785360361Flushing 1588230740/table: creating writer at 1675785360383 (+22 ms)Flushing 1588230740/table: appending metadata at 1675785360385 (+2 ms)Flushing 1588230740/table: closing flushed file at 1675785360385Flushing 1588230740/info: reopening flushed file at 1675785360403 (+18 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675785360408 (+5 ms)Flushing 1588230740/table: reopening flushed file at 1675785360412 (+4 ms)Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 91ms, sequenceid=210, compaction requested=false at 1675785360415 (+3 ms)Writing region close event to WAL at 1675785360417 (+2 ms)Running coprocessor post-close hooks at 1675785360419 (+2 ms)Closed at 1675785360420 (+1 ms) 2023-02-07 15:56:00,420 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-07 15:56:00,473 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 15:56:00,473 INFO [RS:1;jenkins-hbase12:35657] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,35657,1675785236854; zookeeper connection closed. 2023-02-07 15:56:00,473 ERROR [Listener at localhost.localdomain/41965-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@65f8183b rejected from java.util.concurrent.ThreadPoolExecutor@61c7905c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 15:56:00,473 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@651305eb] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@651305eb 2023-02-07 15:56:00,473 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:35657-0x1013e2134d40002, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 15:56:00,473 ERROR [Listener at localhost.localdomain/41965-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@30376b2f rejected from java.util.concurrent.ThreadPoolExecutor@61c7905c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 15:56:00,524 INFO [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,38629,1675785236797; all regions closed. 2023-02-07 15:56:00,524 INFO [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,37399,1675785236898; all regions closed. 2023-02-07 15:56:00,531 DEBUG [RS:0;jenkins-hbase12:38629] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/oldWALs 2023-02-07 15:56:00,531 DEBUG [RS:2;jenkins-hbase12:37399] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/oldWALs 2023-02-07 15:56:00,531 INFO [RS:0;jenkins-hbase12:38629] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C38629%2C1675785236797:(num 1675785238564) 2023-02-07 15:56:00,531 INFO [RS:2;jenkins-hbase12:37399] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C37399%2C1675785236898.meta:.meta(num 1675785238728) 2023-02-07 15:56:00,531 DEBUG [RS:0;jenkins-hbase12:38629] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 15:56:00,531 INFO [RS:0;jenkins-hbase12:38629] regionserver.LeaseManager(133): Closed leases 2023-02-07 15:56:00,531 INFO [RS:0;jenkins-hbase12:38629] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase12:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-07 15:56:00,532 INFO [RS:0;jenkins-hbase12:38629] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-07 15:56:00,532 INFO [regionserver/jenkins-hbase12:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-07 15:56:00,532 INFO [RS:0;jenkins-hbase12:38629] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-07 15:56:00,532 INFO [RS:0;jenkins-hbase12:38629] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-07 15:56:00,534 INFO [RS:0;jenkins-hbase12:38629] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:38629 2023-02-07 15:56:00,536 DEBUG [RS:2;jenkins-hbase12:37399] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/oldWALs 2023-02-07 15:56:00,536 INFO [RS:2;jenkins-hbase12:37399] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C37399%2C1675785236898:(num 1675785238559) 2023-02-07 15:56:00,536 DEBUG [RS:2;jenkins-hbase12:37399] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 15:56:00,536 INFO [RS:2;jenkins-hbase12:37399] regionserver.LeaseManager(133): Closed leases 2023-02-07 15:56:00,536 INFO [RS:2;jenkins-hbase12:37399] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase12:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-07 15:56:00,536 INFO [regionserver/jenkins-hbase12:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-07 15:56:00,537 INFO [RS:2;jenkins-hbase12:37399] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:37399 2023-02-07 15:56:00,541 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-07 15:56:00,541 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:56:00,541 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,38629,1675785236797 2023-02-07 15:56:00,541 ERROR [Listener at localhost.localdomain/41965-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2e71fbe2 rejected from java.util.concurrent.ThreadPoolExecutor@2a4e7af4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 15:56:00,552 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:56:00,552 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,37399,1675785236898 2023-02-07 15:56:00,552 ERROR [Listener at localhost.localdomain/41965-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@36f85cb9 rejected from java.util.concurrent.ThreadPoolExecutor@32c7cebe[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 15:56:00,552 ERROR [Listener at localhost.localdomain/41965-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7f66f785 rejected from java.util.concurrent.ThreadPoolExecutor@2a4e7af4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 15:56:00,562 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase12.apache.org,38629,1675785236797] 2023-02-07 15:56:00,562 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase12.apache.org,38629,1675785236797; numProcessing=2 2023-02-07 15:56:00,583 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase12.apache.org,38629,1675785236797 already deleted, retry=false 2023-02-07 15:56:00,583 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase12.apache.org,38629,1675785236797 expired; onlineServers=1 2023-02-07 15:56:00,583 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase12.apache.org,37399,1675785236898] 2023-02-07 15:56:00,583 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase12.apache.org,37399,1675785236898; numProcessing=3 2023-02-07 15:56:00,594 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase12.apache.org,37399,1675785236898 already deleted, retry=false 2023-02-07 15:56:00,594 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase12.apache.org,37399,1675785236898 expired; onlineServers=0 2023-02-07 15:56:00,594 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,37073,1675785235583' ***** 2023-02-07 15:56:00,594 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-07 15:56:00,595 DEBUG [M:0;jenkins-hbase12:37073] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3e411c8a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-02-07 15:56:00,595 INFO [M:0;jenkins-hbase12:37073] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,37073,1675785235583 2023-02-07 15:56:00,595 INFO [M:0;jenkins-hbase12:37073] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,37073,1675785235583; all regions closed. 2023-02-07 15:56:00,595 DEBUG [M:0;jenkins-hbase12:37073] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 15:56:00,595 DEBUG [M:0;jenkins-hbase12:37073] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-07 15:56:00,595 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-07 15:56:00,595 DEBUG [M:0;jenkins-hbase12:37073] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-07 15:56:00,595 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.small.0-1675785238165] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.small.0-1675785238165,5,FailOnTimeoutGroup] 2023-02-07 15:56:00,595 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.large.0-1675785238163] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.large.0-1675785238163,5,FailOnTimeoutGroup] 2023-02-07 15:56:00,595 INFO [M:0;jenkins-hbase12:37073] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-07 15:56:00,596 INFO [M:0;jenkins-hbase12:37073] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-07 15:56:00,596 INFO [M:0;jenkins-hbase12:37073] hbase.ChoreService(369): Chore service for: master/jenkins-hbase12:0 had [] on shutdown 2023-02-07 15:56:00,596 DEBUG [M:0;jenkins-hbase12:37073] master.HMaster(1502): Stopping service threads 2023-02-07 15:56:00,596 INFO [M:0;jenkins-hbase12:37073] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-07 15:56:00,596 ERROR [M:0;jenkins-hbase12:37073] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-07 15:56:00,596 INFO [M:0;jenkins-hbase12:37073] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 15:56:00,596 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-07 15:56:00,604 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-07 15:56:00,604 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 15:56:00,604 DEBUG [M:0;jenkins-hbase12:37073] zookeeper.ZKUtil(398): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-07 15:56:00,605 WARN [M:0;jenkins-hbase12:37073] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-07 15:56:00,605 INFO [M:0;jenkins-hbase12:37073] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 15:56:00,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-07 15:56:00,605 INFO [M:0;jenkins-hbase12:37073] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-07 15:56:00,605 DEBUG [M:0;jenkins-hbase12:37073] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-07 15:56:00,605 INFO [M:0;jenkins-hbase12:37073] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 15:56:00,605 DEBUG [M:0;jenkins-hbase12:37073] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 15:56:00,605 DEBUG [M:0;jenkins-hbase12:37073] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-07 15:56:00,605 DEBUG [M:0;jenkins-hbase12:37073] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 15:56:00,605 INFO [M:0;jenkins-hbase12:37073] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.81 KB heapSize=719.67 KB 2023-02-07 15:56:00,618 INFO [M:0;jenkins-hbase12:37073] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.81 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d4334d7326e942c797a289f70ecd17d3 2023-02-07 15:56:00,620 INFO [M:0;jenkins-hbase12:37073] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d4334d7326e942c797a289f70ecd17d3 2023-02-07 15:56:00,621 DEBUG [M:0;jenkins-hbase12:37073] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d4334d7326e942c797a289f70ecd17d3 as hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d4334d7326e942c797a289f70ecd17d3 2023-02-07 15:56:00,624 INFO [M:0;jenkins-hbase12:37073] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d4334d7326e942c797a289f70ecd17d3 2023-02-07 15:56:00,624 INFO [M:0;jenkins-hbase12:37073] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43555/user/jenkins/test-data/3282647d-6c18-1df7-2c52-0ad7a7aa78e8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d4334d7326e942c797a289f70ecd17d3, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-07 15:56:00,624 INFO [M:0;jenkins-hbase12:37073] regionserver.HRegion(2947): Finished flush of dataSize ~608.81 KB/623418, heapSize ~719.66 KB/736928, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 19ms, sequenceid=1282, compaction requested=false 2023-02-07 15:56:00,625 INFO [M:0;jenkins-hbase12:37073] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 15:56:00,625 DEBUG [M:0;jenkins-hbase12:37073] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675785360605Disabling compacts and flushes for region at 1675785360605Disabling writes for close at 1675785360605Obtaining lock to block concurrent updates at 1675785360605Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675785360605Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=623418, getHeapSize=736928, getOffHeapSize=0, getCellsCount=1499 at 1675785360605Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675785360606 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675785360606Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675785360609 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675785360609Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675785360621 (+12 ms)Finished flush of dataSize ~608.81 KB/623418, heapSize ~719.66 KB/736928, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 19ms, sequenceid=1282, compaction requested=false at 1675785360624 (+3 ms)Writing region close event to WAL at 1675785360625 (+1 ms)Closed at 1675785360625 2023-02-07 15:56:00,628 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-07 15:56:00,628 INFO [M:0;jenkins-hbase12:37073] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-07 15:56:00,629 INFO [M:0;jenkins-hbase12:37073] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:37073 2023-02-07 15:56:00,630 ERROR [BP-1532654085-136.243.104.168-1675785230655 heartbeating to localhost.localdomain/127.0.0.1:43555] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1532654085-136.243.104.168-1675785230655 (Datanode Uuid 8f3909c3-b4fb-4933-8c65-751b48eaa405) service to localhost.localdomain/127.0.0.1:43555 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 15:56:00,636 DEBUG [M:0;jenkins-hbase12:37073] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase12.apache.org,37073,1675785235583 already deleted, retry=false 2023-02-07 15:56:00,663 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 15:56:00,663 INFO [RS:0;jenkins-hbase12:38629] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,38629,1675785236797; zookeeper connection closed. 2023-02-07 15:56:00,663 ERROR [Listener at localhost.localdomain/41965-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7a9e583 rejected from java.util.concurrent.ThreadPoolExecutor@2a4e7af4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 15:56:00,664 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:38629-0x1013e2134d40001, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 15:56:00,664 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@12dc93cf] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@12dc93cf 2023-02-07 15:56:00,664 ERROR [Listener at localhost.localdomain/41965-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@47f11a20 rejected from java.util.concurrent.ThreadPoolExecutor@2a4e7af4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 15:56:00,763 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 15:56:00,763 INFO [M:0;jenkins-hbase12:37073] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,37073,1675785235583; zookeeper connection closed. 2023-02-07 15:56:00,763 ERROR [Listener at localhost.localdomain/41965-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@22d77a0c rejected from java.util.concurrent.ThreadPoolExecutor@148c2fde[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 15:56:00,764 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): master:37073-0x1013e2134d40000, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 15:56:00,764 ERROR [Listener at localhost.localdomain/41965-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@631142c2 rejected from java.util.concurrent.ThreadPoolExecutor@148c2fde[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 15:56:00,864 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 15:56:00,864 INFO [RS:2;jenkins-hbase12:37399] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,37399,1675785236898; zookeeper connection closed. 2023-02-07 15:56:00,864 ERROR [Listener at localhost.localdomain/41965-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@32118c61 rejected from java.util.concurrent.ThreadPoolExecutor@32c7cebe[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 15:56:00,864 DEBUG [Listener at localhost.localdomain/41965-EventThread] zookeeper.ZKWatcher(600): regionserver:37399-0x1013e2134d40003, quorum=127.0.0.1:51095, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 15:56:00,864 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@24ef3d57] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@24ef3d57 2023-02-07 15:56:00,864 ERROR [Listener at localhost.localdomain/41965-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4a468741 rejected from java.util.concurrent.ThreadPoolExecutor@32c7cebe[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 15:56:00,865 INFO [Listener at localhost.localdomain/41965] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-07 15:56:00,866 WARN [Listener at localhost.localdomain/41965] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-07 15:56:00,872 INFO [Listener at localhost.localdomain/41965] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-07 15:56:00,980 WARN [BP-1532654085-136.243.104.168-1675785230655 heartbeating to localhost.localdomain/127.0.0.1:43555] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1532654085-136.243.104.168-1675785230655 (Datanode Uuid bb8a498b-c978-4fc3-b7cd-17da28adf753) service to localhost.localdomain/127.0.0.1:43555 2023-02-07 15:56:00,982 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/dfs/data/data5/current/BP-1532654085-136.243.104.168-1675785230655] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-07 15:56:00,982 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/dfs/data/data6/current/BP-1532654085-136.243.104.168-1675785230655] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-07 15:56:00,984 WARN [Listener at localhost.localdomain/41965] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-07 15:56:00,987 INFO [Listener at localhost.localdomain/41965] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-07 15:56:01,095 WARN [BP-1532654085-136.243.104.168-1675785230655 heartbeating to localhost.localdomain/127.0.0.1:43555] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1532654085-136.243.104.168-1675785230655 (Datanode Uuid 8f3909c3-b4fb-4933-8c65-751b48eaa405) service to localhost.localdomain/127.0.0.1:43555 2023-02-07 15:56:01,097 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/dfs/data/data3/current/BP-1532654085-136.243.104.168-1675785230655] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-07 15:56:01,098 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/dfs/data/data4/current/BP-1532654085-136.243.104.168-1675785230655] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-07 15:56:01,101 WARN [Listener at localhost.localdomain/41965] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-07 15:56:01,104 INFO [Listener at localhost.localdomain/41965] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-07 15:56:01,214 WARN [BP-1532654085-136.243.104.168-1675785230655 heartbeating to localhost.localdomain/127.0.0.1:43555] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-07 15:56:01,216 WARN [BP-1532654085-136.243.104.168-1675785230655 heartbeating to localhost.localdomain/127.0.0.1:43555] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1532654085-136.243.104.168-1675785230655 (Datanode Uuid 7f437b77-6c8e-4bcb-8556-91fa3c36506b) service to localhost.localdomain/127.0.0.1:43555 2023-02-07 15:56:01,217 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/dfs/data/data1/current/BP-1532654085-136.243.104.168-1675785230655] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-07 15:56:01,217 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6ca4ca6-6a28-9758-d759-bd2ce1f9e3ed/cluster_55a7a609-bc85-e7a3-ce4b-8cacda96471a/dfs/data/data2/current/BP-1532654085-136.243.104.168-1675785230655] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-07 15:56:01,236 INFO [Listener at localhost.localdomain/41965] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-02-07 15:56:01,417 INFO [Listener at localhost.localdomain/41965] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-07 15:56:01,639 INFO [Listener at localhost.localdomain/41965] hbase.HBaseTestingUtility(1293): Minicluster is down